
A DELPHI EXPERT ASSESSMENT OF PROFESSIONAL 
CERTIFICATION PROGRAMS FOR CONTRACTING PERSONNEL 

THESIS 


Ronald L. Tougaw, Jr., Captain, USAF 


AFIT/GAQ/ENV/01M-15 


DEPARTMENT OF THE AIR FORCE 
AIR UNIVERSITY 

AIR FORCE INSTITUTE OF TECHNOLOGY 


Wright-Patterson Air Force Base, Ohio 


APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITED. 


20010612 124 



The views expressed in this thesis are those of the author and do not reflect the official 
policy or position of the United States Air Force, Department of Defense, or the U. S. 
Government. 




AFIT/GAQ/ENV/01M-15 


A DELPHI EXPERT ASSESSMENT OF PROFESSIONAL CERTIFICATION 
PROGRAMS FOR CONTRACTING PERSONNEL 

THESIS 


Presented to the Faculty 

Department of Systems and Engineering Management 
Graduate School of Engineering and Management 
Air Force Institute of Technology 
Air University 

Air Education and Training Command 
In Partial fulfillment of the Requirements for the 
Degree of Master of Science of in Acquisition Management 


Ronald L. Tougaw Jr., B.A. 
Captain, USAF 

March 2001 


APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITED 




AFIT/GAQ/ENY/01M-15 


A DELPHI EXPERT ASSESSMENT OF PROFESSIONAL CERTIFICATION 
PROGRAMS FOR CONTRACTING PERSONNEL 


Ronald L. Tougaw Jr., B.A. 
Captain, USAF 


Approved: 



Paul W. Thurston, Jr., Major, USAF (Co-Chairman) date 



Acknowledgements 


I would like to express my sincere appreciation to my faculty advisor. Major Paul 
Thurston, for his guidance and support throughout the course of this thesis effort. The 
successful completion of this report is largely a result of his insight, encouragement and 
leadership; I am forever in his debt. I would also like to recognize the other members of 
my advisory committee, Lt Col Bradley Ayres, Lt Col David Petrillo and Major Michael 
Rehg, for their contribution to this work. 

Special thanks are due to the members of the expert panels for offering their time and 
sharing their experiences in the midst of demanding schedules and other commitments. 
This research effort would not have been possible without their participation. 

Finally, to all those who nurtured me over the past eighteen months, beyond the 
realm of academia, I cannot overstate my gratitude. To my wife,MBH- for her love, 
support, and faith ... for life. To Jeff Payne - for renewing my sanity every weekend. 

To Bill E., Bill W., Dan, Doug, John, and Tommy - for the back row antics. To all 
veterans of Operation Get Porky — you are not forgotten. 


Ronald L. Tougaw, Jr. 




Table of Contents 


Page 

Acknowledgements.iv 

List of Tables.x 

Abstract.xiii 

I. Introduction.1 

Background.1 

Purpose.2 

Research Questions.2 

Scope.3 

II. Literature Review.5 

Introduction.5 

Purpose of Professional Certification.5 

The Contracting Career Field.9 

Certification Requirements.10 

DoD Acquisition Career Development Program.10 

National Contract Management Association (NCMA).12 

National Association of Purchasing Management (NAPM).16 

National Institute of Governmental Purchasing (NIGP).18 

Measured Qualifications for Certification.21 

Summary.23 

III. Methodology.25 

Overview.25 

Basis.25 

Delphi.26 

Research Design.30 

Selecting Experts.30 

Cycles of Discussion.31 

Assessing the Certification Programs.33 

IV. Findings and Analysis.34 

34 


Overview 































Page 


Delphi Cycle 1.34 

Question 1.35 

Civilian Responses.35 

Military Responses.36 

Question 2.36 

Civilian Responses.37 

Military Responses.38 

Question 3.38 

Civilian Responses.38 

Military Responses.40 

Question 4.41 

Civilian Responses.41 

Military Responses.41 

Question 5.41 

Civilian Responses.42 

Military Responses.42 

Question 6.43 

Civilian Responses.43 

Military Responses.44 

Question 7.44 

Civilian Responses.44 

Military Responses.45 

Cross-case Comparison.46 

Delphi Cycle 2.47 

Civilian Responses for Cycle 2a.47 

Question 1.48 

Question 2.49 

Question 3.49 

Question 4.50 

Military Responses for Cycle 2a.51 

Question 1.51 

Question 2.51 

Question 3.53 

Question 4.53 

Civilian Responses for Cycle 2b.54 

Question 1.54 

Question 2.55 

Question 3.55 

Question 4.56 

Military Responses for Cycle 2b.57 

Question 1.57 

Question 2.58 


vi 












































Page 


Question 3.58 

Question 4.59 

Civilian Responses for Cycle 2c.59 

Question 1.60 

Question 2.60 

Military Responses for Cycle 2c.61 

Question 1.61 

Question 2.62 

Civilian Responses for Cycle 2d.62 

Question .63 

Military Responses for Cycle 2d.63 

Question .63 

Cross-case Comparison.64 

Delphi Cycle 3.65 

Question 1.66 

Question 2.67 

Questions 3 and 4.67 

Question 5.68 

Question 6.69 

Question 7.69 

Questions 8 and 9.70 

Question 10.70 

Question 11.71 

Question 12.71 

Question 13.72 

Delphi Cycle 4.72 

Summary.73 

V. Conclusions and Recommendations.75 

Overview.75 

Conclusions.75 

Research Question 1.75 

Research Question 2.76 

Recommendations.78 

Certified Professional Contracts Manager (CPCM) Program.78 

Acquisition Professional Development Program.78 

Formal Education.78 

Experience.80 

Training.80 

Comprehensive Examination.81 

Re-certification.82 

Summary.82 


Vll 












































Page 

Limitations.83 

Recommendations for Future Research.84 

Summary.86 

Appendix A: Expert Panel Membership.87 

Appendix B: Initial Information for Participants.88 

Appendix C. Delphi - Cycle 1 - Initial Questionnaire.91 

Appendix D. Synopsis of Responses For Delphi Cycle 1.93 

Appendix E. Delphi - Cycle 2(a) - Formal Education (Civilian).109 

Appendix F. Delphi - Cycle 2(a) - Formal Education (Military).Ill 

Appendix G. Synopsis of Responses For Delphi Cycle 2a.114 

Appendix H. Delphi - Cycle 2(b) - Training (Civilian).122 

Appendix I. Delphi - Cycle 2(b) - Training (Military).124 

Appendix J. Synopsis of Responses For Delphi Cycle 2b.127 

Appendix K. Delphi - Cycle 2(c) - Experience (Civilian).133 

Appendix L. Delphi - Cycle 2(c) - Experience (Military).135 

Appendix M. Synopsis of Responses For Delphi Cycle 2c.137 

Appendix N. Delphi - Cycle 2(d) - Comprehensive Examination (Civilian).141 

Appendix O. Delphi - Cycle 2(d) - Comprehensive Examination (Military).143 

Appendix P. Synopsis of Responses For Delphi Cycle 2d.145 

Appendix Q. Delphi - Cycle 3.147 

Appendix R. Synopsis of Responses For Delphi Cycle 3.153 

viii 























Page 


Appendix S. Delphi - Cycle 4 - Recommend Changes to Current Programs.168 

Appendix T. Synopsis of Responses For Delphi Cycle 4.171 

Bibliography.174 

Vita.177 


IX 







List of Tables 


Table Page 

1. ACDP / APDP Certification Requirements.13 

2. CPCM Certification Requirements.15 

3. CACM Certification Requirements.16 

4. NAPM Certification Requirements.18 

5. Certified Public Purchasing Buyer (CPPB) Eligibility Requirements.20 

6. Certified Public Purchasing Officer (CPPO) Eligibility Requirements.21 

7. Certification Requirements Compared.22 

8. Civilian Responses to Delphi Cycle 1 / Question 1.93 

9. Military Responses to Delphi Cycle 1 / Question 1.94 

10. Civilian Responses to Delphi Cycle 1 / Question 2.96 

11. Military Responses to Delphi Cycle 1 / Question 2.97 

12. Civilian Responses to Delphi Cycle 1 / Question 3.98 

13. Military Responses to Delphi Cycle 1 / Question 3.99 

14. Civilian Responses to Delphi Cycle 1 / Question 4.101 

15. Military Responses to Delphi Cycle 1 / Question 4.102 

16. Civilian Responses to Delphi Cycle 1 / Question 5.103 

17. Military Responses to Delphi Cycle 1 / Question 5.104 

18. Civilian Responses to Delphi Cycle 1 / Question 6.105 

19. Military Responses to Delphi Cycle 1 / Question 6.106 

20. Civilian Responses to Delphi Cycle 1 / Question 7.107 


x 























Page 

21. Military Responses to Delphi Cycle 1 / Question 7.108 

22. Civilian Responses to Delphi Cycle 2a / Question 1.114 

23. Military Responses to Delphi Cycle 2a / Question 1.115 

24. Civilian Responses to Delphi Cycle 2a / Question 2.116 

25. Military Responses to Delphi Cycle 2a / Question 2.117 

26. Civilian Responses to Delphi Cycle 2a / Question 3.118 

27. Military Responses to Delphi Cycle 2a / Question 3.119 

28. Civilian Responses to Delphi Cycle 2a / Question 4.120 

29. Military Responses to Delphi Cycle 2a / Question 4.121 

30. Civilian Responses to Delphi Cycle 2b / Question 1.126 

31. Military Responses to Delphi Cycle 2b / Question 1.127 

32. Civilian Responses to Delphi Cycle 2b / Question 2.128 

33. Military Responses to Delphi Cycle 2b / Question 2.128 

34. Civilian Responses to Delphi Cycle 2b / Question 3.129 

35. Military Responses to Delphi Cycle 2b / Question 3.130 

36. Civilian Responses to Delphi Cycle 2b / Question 4.131 

37. Military Responses to Delphi Cycle 2b / Question 4.132 

38. Civilian Responses to Delphi Cycle 2c / Question 1.137 

39. Military Responses to Delphi Cycle 2c / Question 1.138 

40. Civilian Responses to Delphi Cycle 2c / Question 2.139 

41. Military Responses to Delphi Cycle 2c / Question 2.140 

42. Civilian Responses to Delphi Cycle 2d.145 


xi 

























Page 


43. Military Responses to Delphi Cycle 2d.146 

44. Responses to Delphi Cycle 3 / Question 1.153 

45. Responses to Delphi Cycle 3 / Question 2.154 

46. Responses to Delphi Cycle 3 / Question 3.156 

47. Responses to Delphi Cycle 3 / Question 4.157 

48. Responses to Delphi Cycle 3 / Question 5.158 

49. Responses to Delphi Cycle 3 / Question 6.160 

50. Responses to Delphi Cycle 3 / Question 7.161 

51. Responses to Delphi Cycle 3 / Question 8.162 

52. Responses to Delphi Cycle 3 / Question 9.163 

53. Responses to Delphi Cycle 3 / Question 10.164 

54. Responses to Delphi Cycle 3 / Question 11.165 

55. Responses to Delphi Cycle 3 / Question 12.166 

56. Responses to Delphi Cycle 3 / Question 13.167 

57. Recommended Changes to Air Force APDP Certification Requirements.171 

58. Recommended Changes to NCMA CPCM Certification Requirements.173 


xii 



















AFIT/GAQ/EN V/01M-15 


Abstract 

This study examined the strategic goals that were the basis for the DoDs 
implementation of mandatory certification for individuals serving in the contracting 
career field of the acquisition workforce. The study then went on to enlist panels of 
contracting experts to assess the extent to which two of the available certification 
programs meet, or fail to meet, those goals. 

The research method employed to gather input from experts was a Delphi discussion 
technique. Two separate panels, one consisting of Air Force civilian employees, the 
other of Air Force active duty officers, participated in eight Delphi iterations facilitated 
by the researcher. The initial iteration required the experts to answer seven open-ended 
questions regarding certification requirements. Once the researcher confirmed the 
experts were focusing on the strategic value of certification, the subsequent iterations 
were used to identify and allow comment on areas of disagreement both within and 
between groups. 

The primary impact of this study was that it identified many important issues that 
require attention before the certification programs will be able to fulfill the strategic 
objectives. Overall, the experts believed the NCMA CPCM to be a truly professional 
certification, indicative of an individual’s professional status and level of development, 
while the Air Force’s APDP certification process was seen as more of a tracking device 
for training and education rather than a program that enhances professional development. 

xiii 




A DELPHI EXPERT ASSESSMENT OF PROFESSIONAL CERTIFICATION 


PROGRAMS FOR CONTRACTING PERSONNEL 

I. Introduction 

Background 

In 1999, Lorton and Dunn conducted a study to detennine potential deficiencies in 
existing professional certification programs for DoD contracting personnel and develop 
an “ideal” certification model to overcome those deficiencies (Lorton and Dunn, 1999: 

3). This thesis is a continuance and expanse of their research. 

The Defense Acquisition Workforce Improvement Act (DAWIA) of 1990 resulted in 
the establishment of the Defense Acquisition University (DAU) and the prescription of 
specific training requirements for acquisition personnel (United States Congress, 1990). 
DAU began certifying DoD acquisition professionals at three different levels based on an 
individual’s education, experience, and completion of required acquisition courses. The 
DAWIA education and training requirements are applicable to both civilian and military 
members of the DoD acquisition workforce; many permanent and deployed positions 
require the assigned individual to possess a minimum level of certification. This 
heightened focus on professional development led DoD acquisition leadership to 
encourage personnel to pursue certifications offered by other organizations as well. The 
National Institute of Governmental Purchasing (NIGP), National Association of 
Purchasing Management (NAPM), and National Contract Management Association 
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(NCMA) all offer various professional certifications for both government and private 
sector acquisition professionals. 


Purpose 

Professional certifications serve several purposes for the individuals, organizations, 
and clientele involved in various vocational fields. Certifications enable organizations to 
identify current and prospective employees with the highest level of proficiency, 
incentivize employees to maintain the required level of skills and education, provide a 
measure of accountability for human resource strategies, and service the public trust by 
ensuring qualification for practice. This research effort evaluated two of the existing 
contracting professional certification methods available to DoD personnel in order to 
determine the extent to which the requirements of those methods are sufficient to meet 
the strategic initiatives behind the certification process, and assess the reasons attributable 
to that success or failure. 


Research Questions 

The previous study focused on five specific elements of the certification process to 
detennine the appropriate structure of an ideal contracting certification program. This 
study took a strategic standpoint on the subject by attempting to answer two basic 
questions: 

1. What are the strategic goals for certifying acquisition professionals? 

2. To what extent do Air Force acquisition experts believe current certification 

requirements meet or fail to meet those goals? 
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Although the purpose of this study does not include the design of new certification 
requirements, recommendations for improvement to the existing certification programs 
were made, based on the findings in answering the above questions. 

Scope 

The DoD acquisition workforce consists of eleven distinct career fields each with 
specific certification requirements (DAU, 1999: 27). Although the results of this study 
may be of value to professionals in several of the career fields, a review of each separate 
certification program would require a greater amount of time than is available for this 
research effort. In order to provide a comprehensive analysis of the research questions, 
this study is limited to assessing the certification programs for the contracting career 
field. 

As previously discussed, several organizations offer certifications to contracting 
professionals. The requirements of DAWIA are mandated across the DoD; however, 
certification from NIGP, NAPM, and NCMA are optional. Of these latter certifications, 
those offered by NCMA are the mostly widely sought after by DoD contracting 
personnel. For this reason, this thesis answered the research questions as they applied to 
the DAWIA and NCMA certification requirements. The results of this study should not 
be translated as an assessment of all available certification programs. 

The certification programs discussed in this study are applicable to members in all 
branches of the armed forces. Although the various components differ in the execution 
of specific contracting functions, all are subject to the certification requirements instituted 
through DAWIA and, as acquisition professionals, all are afforded the opportunity to 
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seek certification from NCMA. However, in order to foster a prolonged discussion 
between experts, this study was limited to two discussion groups from the Air Force’s 
contracting career field. Both active duty military and civilian members were invited to 
participate. 


4 


II. Literature Review 


Introduction 

This chapter discusses the relevant literature reviewed in preparation for this study. 
The first portion of the chapter establishes the role of professional certification and the 
benefits realized by certified individuals, certifying associations, and the profession in 
general. Next, a brief definition of the contracting career field presents a context for 
applying the literature to this study. The chapter concludes with a review of existing 
certification requirements for those programs available to the Air Force’s acquisition 
workforce, specifically the contracting career field, and a summary of the common and 
unique characteristics among the programs. 

Purpose of Professional Certification 

The Air Force’s requirement for all acquisition professionals to be certified through 
the Acquisition Professional Development Program (APDP) came in direct response to 
the Defense Acquisition Workforce Improvement Act (DAWIA), implying that the 
program works toward the betterment of the workforce. Professional certifications have 
two goals at their foundation: (1) the promotion of competencies by measuring 
distinctions between the skills of experienced practitioners and entry-level employees; 
and (2) enhancing the profession by standardizing the proficiency required before an 
individual can obtain professional status and the requirements to be met before one can 
enter the field of practice (Wiley, 1995: 271). Certification requirements include various 
levels of training, education, and experience. Intuitively, it is not hard to imagine that 
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individuals would find value added to their personal careers by receiving advanced skills 
through the completion of certification requirements. However, there is some concern as 
to whether professional certification programs also meet the strategic goals for which 
they were intended. 

Certification “indicates that an individual has attained professional standing in a 
given field” (Keating, 1998: 51). Individuals designated by a certification enjoy a variety 
of opportunities for career progression. Although most organizations do not require all 
employees to receive certification, there are often policies that preclude certain positions 
from being held by other than a certified individual. The National Association of 
Purchasing Management (NAPM) vice president of certification was quoted as saying, 
“Certainly a reason for the C.P.M. is to advance a career. Holding that credential will 
help people with their career path” (Ciancarelli, 1999: 114). A salary survey conducted 
by Purchasing Magazine reported a 26 percent higher salary for purchasing professionals 
with a C.P.M. certification (Ciancarelli, 1999: 114). 

In addition to higher pay, certification programs offer individuals a valuable 
networking resource (Patterson, 1997: 64; Ciancarelli, 1999: 114). Often, members of 
various organizations seeking the same certification attend training courses and seminars 
together. At these courses, students can learn as much from each other as they do from 
the instruction material. In the course of pursuing a professional certification, individuals 
meet future employers, employees, or business partners they might not have contacted 
otherwise. 

Beyond the benefit to individuals, certifications also serve organizations. Certifying 
associations experience an increase in membership as the recognition of the profession 
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increases. These associations can also gain substantial non-due revenue through the 
administration of a professional certification program (Peluso, 2000: 65). Purchasing 
associations charge fees as high as $540 for certification applications as well as periodic 
fees to reestablish a current certification. The revenue and notoriety provided by a 
respected certification program are essential to the association’s continued ability to 
provide professional development resources to its members. 

If certification programs benefited only individual careers and the administering 
associations, it would be unlikely an organization such as the DoD would place strategic 
emphasis on the certification of its workforce. But DAWIA and APDP did have a 
strategic intent as the basis for their inception. In Sociology of Occupations and 
Professions . Pavalko found certain qualities attributable to professions: 

(1) A unique knowledge base justifying the claim to special expertise. 

(2) A long training period requiring specialized knowledge and indoctrination 
into the occupational culture. 

(3) Relevance of work to social values. 

(4) A service versus a profit motivation. 

(5) Occupational autonomy. The profession is self-regulating and self¬ 
controlling. Only members of the profession judge and certify who is 
competent to practice. 

(6) A strong sense of commitment and loyalty to the profession. 

(7) A strong sense of identity resulting in a significant subculture. 

(8) A code of ethics and system of norms that are parts of the subculture, 
reinforcing motivation, autonomy, and commitment. (Snider, 1996: 98) 

Through certification, professions can express collective ideals, strengthen the sense of 

those ideals and responsibility among its members, improve practice, and raise awareness 
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of the profession’s technical contribution to society and its framework of values (Imrey, 
1994: 69). DAWIA intended to professionalize the acquisition workforce by “treating 
them as though acquisition has some of the attributes of a profession” (Snider, 1996: 99). 
Acquisition has been determined to require a unique knowledge base and specialized 
training and experience. “The workforce then becomes professional by meeting the 
requirements for acquisition education, training, experience, and tenure provided for 
under DAWIA” (Snider, 1996: 99). The purpose of DAWIA, and the certification 
programs established by DAWIA, was to enhance the professionalism of the acquisition 
workforce, those individuals doing business in a career field that possesses the qualities 
of a profession. While the efforts to do so have to occur on an individual basis, this 
investment in employee development was expected to result in returns of improvement to 
the overall acquisition process for the DoD (Snider, 1996: 100). DAWIA did not simply 
offer a service to members of the acquisition workforce, it was a strategic initiative aimed 
at improving the manner in which the DoD conducts the business of acquisition. The 
numerous reforms to the acquisition process over the past decade require an able, 
professional workforce for successful implementation. 

While the intention of DAWIA and the basis for the APDP certification program is 
strategic in nature, there is some concern regarding the execution of the certification 
program and its ability to meet those strategic goals. Establishing a professional 
certification requires the development of some minimum standard to qualify an 
individual for a certificate. A senior vice president at Ronin Enterprises asks, “Do 
certification processes enhance responsibility, or does the existence of external, and 
usually minimal standards which give us a ‘passing grade’, contribute to the decline of 
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the very professionalism they purport to support (Levit, 1995: 292)?” An instructor at 
Defense Acquisition University (DAU) witnessed more senior civilian and military 
officer students who gave their reasons for attending the course in terms of “filling a 
square” on their certification application than those who cited professionalism and 
developmental purposes (Snider, 1996: 97). DAWLA and its certification programs are 
well intentioned in seeking to fashion a professional workforce; however, it might be 
argued that DAWLA is only one point on the path to professionalism and that some level 
of cultural change will be necessary for the organization to fully realize its potential 
strategic benefits (Snider, 1996: 103). 

Professional certifications, including that managed within the Air Force, support 
multiple motivations - some organizational and some personal. The strategic goals of 
process improvement, reform, and efficiency must rely on the development of the 
individuals who carry out those initiatives. The personal, careerist goals of individuals 
are what Snider terms to be the “price of professionalism,” a necessary side effect of 
improving a profession through certification. However, there is still some question as to 
which goals, the strategic or personal, take precedence within the program. 

The Contracting Career Field 

Chapter I established the scope of this study as encompassing the certification 
programs for individuals in the Air Force’s contracting career field. The contracting 
career field is described as follows: 

The contracting career field includes the positions of contract negotiator, 
contract specialist, contract termination specialist, contract administrator, 
procurement analyst, administrative contracting officer, procuring contracting 
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officer, contract price and/or cost analyst, contracting officer, and termination 
contracting officer. Individuals in this career field develop, manage, supervise, 
or perform procedures involving the procurement of supplies and services; 
construction, research, and development; acquisition planning; cost and price 
analysis; selection and solicitation of sources; preparation, negotiation, and 
award of contracts; all phases of contract administration; and termination, or 
close out of contracts. The employee is required to have knowledge of the 
legislation, policies, regulations, and methods used in contracting, as well as 
knowledge of business and industry practices, sources of supply, cost factors, 
cost and price analysis techniques, and general requirements characteristics. 
(Defense Acquisition University, 1999: 28-29) 

The sections that follow describe specific certification programs as they apply to 

individuals in the contracting career field. 


Certification Requirements 

DoD Acquisition Career Development Program. The Career Development Program for 
Acquisition Personnel (CDPAP) was established in response to DAWIA wherein the 
Under Secretary of Defense (Acquisition) (USD(A)) is required to “establish education, 
training, and experience standards for each acquisition position based on the level of 
complexity of duties carried out in that position” (Department of Defense, 1991: 2). In 
1995, the CDPAP was replaced by the DoD Acquisition Career Development Program 
(ACDP), under the Under Secretary of Defense for Acquisition and Technology 
(USD(A&T)) (Department of Defense, 1995: i). The purpose of the ACDP is to develop 
a highly qualified acquisition workforce, on a long-term basis, while increasing the 
proficiency of the current workforce; thereby improving overall management and 
professionalism and providing opportunities for individuals to broaden their experience 
and progress in their careers (Department of Defense, 1995: 1-1). 
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DoD Acquisition personnel are required to possess the appropriate level of training, 
education, and experience required by their position. The ACDP established both 
mandatory and desired standards for each acquisition career field and addressed certain 
statutory requirements for particular positions. These additional statutory requirements 
are not relegated to all positions and vary between those positions having those 
requirements and, therefore, will not be addressed in this study of the certification 
programs in general. 

The contracting career field is divided into three career levels for the purpose of 
certification: basic or entry (Level I: GS-05 through GS-09, officer 0-1 through 0-3); 
intermediate or journeyman (Level II: GS-09 through GS-12, officer 0-3 through 0-4); 
and advanced or senior (Level III: GS-13 and above, officer 0-4 and above) (Department 
of Defense, 1995: 2-1). The grade levels represented above are provided to assist in 
career development planning but vary and may include enlisted grades. For example, 
many Air Force deployment positions are filled by enlisted members and require a 
minimum certification level. All DoD acquisition personnel are encouraged to pursue the 
next level of certification regardless of their grade or position; however, training quotas 
will give preference to those in current positions requiring the next level of certification. 
The following excerpt from DoD 5000.52-M explains each level of certification: 

Basic or Entry Level (Level I) 

(1) Basic level training standards are designed to establish fundamental 
qualifications and expertise in the individual’s acquisition career field. Development at 
the basic level lays the foundation for career progression and is designed to prepare 
qualified, motivated personnel for positions of increasing responsibility. 

(2) At the basic level, individuals should be exposed to the functions of 
acquisition and the roles of its various specializations. Besides participating in 
education and training courses, it is extremely beneficial for individuals to develop 
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additional and enhanced capabilities through structured on-the-job rotational 
assignments between a variety of functional offices. 

Intermediate or Journeyman Level (Level II) 

(1) At the intermediate level, specialization is initially emphasized. Development 
continues, including rotational assignments, but the responsibilities and length of time 
an individual spends in each position generally increase. 

(2) While specialization is emphasized at the beginning of this level, the 
individual should later begin to broaden his or her background toward a more general 
expertise in the overall processes of his or her acquisition career field. Development of 
the generalist normally involves establishing a good foundation of experience in the 
employee’s primary specialty followed by multi-functional experience accomplished by 
lateral movement to a related acquisition specialty. 

Advanced or Senior Level (Level III) 

At the senior level, advanced acquisition education and training become essential. 
Individuals at this level should seek acquisition related opportunities available through 
the DoD Components, or pursue other formal education opportunities. 

(Department of Defense, 1995: 2-2) 


To ensure each individual member of the acquisition workforce meets the 
requirements of their career level, the ACDP established certification standards. The 
certification standards prescribed by DoD 5000.52-M are DoD wide and implemented 
without supplementation by individual DoD Components (Department of Defense, 1995: 
2-4). Individual components are responsible for implementing and managing the ACDP 
internally; to fulfill that purpose the Air Force instituted the Acquisition Professional 
Development Program (APDP). The current certification standards of each career level 
for contracting personnel are given in Table 1; there are currently no requirements for re¬ 
certification. 


National Contract Management Association (NCMA). The other certification program 
evaluated during this study was that offered by NCMA. NCMA is an individual- 
membership society for contracting professionals whose officers are elected volunteers. 
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Table 1. ACDP / APDP Certification Requirements 


CAREER 

LEVEL 

EDUCATION 

EXPERIENCE 

TRAINING 

Entry 

Level (I) 

Have ONE of: 

Baccalaureate Degree; 

1 year in 
contracting 

Basics of 
Contracting 
(CON 101) 


24 Semester Hours accounting, law, 
business finance, contracts, 
purchasing, economics, industrial 
management, marketing, quantitative 
methods, or organization and 
management 


Principles of 
Contract 

Pricing(CON 104) 


10 years contracting experience as of 1 
Oct 91 



Intermediate 
Level (II) 

Same as Level I 

(Desired) Graduate studies in business 
administration or procurement 

2 years in 
contracting 

(Desired) 2 
additional years 
in contracting 

Intermediate 
Contracting 
(CON 202) 

Intermediate 

Contract 

Pricing(CON 204) 




Government 
Contract Law 
(CON 210) 

Advanced 
Level (III) 

Same as Level I 

(Desired) Masters degree in Business 
Administration or Procurement 

4 years in 
contracting 

(Desired) 4 
additional years 
in contracting 

Executive 
Contracting 
(CON 301) 

Management for 
Contracting 
Supervisors 
(CON 333) 


Note. Entries in the “Training” column denote DAU course titles. Table content derived from 
the Defense Acquisition University Fiscal Year 2000 Course Catalog, Appendix C. 
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NCMA was founded in 1959 to “foster the professional growth and educational 
advancement of its members” (NCMA web site, 2000). In an effort to be the 
“preeminent source of professional development for contract managers,” NCMA 
developed two certifications available to both members and non-members: the Certified 
Associates Contracts Manager (CACM) and the Certified Professional Contracts 
Manager (CPCM). NCMA states that these programs exist to distinguish individuals 
demonstrating professional growth and development in contract and business 
management (NCMA, 2000: 4). The Association believes certification benefits both the 
professional and the profession by enhancing individual development, assuring 
employers of an individual’s qualification, and providing the professional community 
(presumably, clients included) with a measure of excellence (NCMA, 2000: 4). Some 
acquisition professionals view the CACM as a step toward achieving the CPCM, while 
others believe the CACM to be the top certification offered by NCMA; the association 
does not claim a preference for one certification over the other (NCMA, 2000b: 5). The 
CPCM certification recognizes individuals demonstrating a high level of knowledge and 
experience in contracting and other procurement disciplines. Although not specifically a 
Government acquisition certification, the DoD encourages the acquisition workforce to 
pursue the CPCM as well as the CACM. The requirements for obtaining the CPCM are 
detailed in Table 2. 

The CPCM examination is an all essay test consisting of two sessions. The first 
session tests the individual’s knowledge of general contracting principles, while the 
second session is concentrated in seven specific areas: legal; finance, economics, and 
accounting; production; contracting; logistics management; commercial purchasing; and 
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state and local government procurement (NCMA, 2000: 9). Re-certification is required 
every five years through the completion of a total of 60 hours of instruction, 10 of which 


Table 2. CPCM Certification Requirements 


FORMAL 

EDUCATION 

RELEVANT 

EXPERIENCE 

TRAINING 

EXAMINATION 

Bachelor’s Degree 

2 years experience in: 

8 non-credit, 

Completion of 


procurement, legal, MIS/IT, 
inventory mgt, project mgt, 
production, R&D, 
marketing, QA, finance, 
business mgt 

certificate, or 
continuing education 
courses 

CPCM exam 


Note. Formal education requirements are waiverable on a case-by-case basis if the individual 
possesses at least 2 years of college and 10 years relevant experience, while the experience 
requirement is waiverable if the individual possesses a bachelor’s degree. Applicants must verify 
formal education, experience, and training requirements are met to be eligible for the 
examination. Table content derived from CPCM Information Packet, 5th Edition. 


must be within the 18 months prior to re-certification (NCMA, 2000: 15). If this 
continuing education is met within the five-year deadline, the examination need not be 
retaken. 

The CACM was initiated in 1980 and recognizes mastery in the fundamentals 
specific to Government acquisition (NCMA, 2000b: 4-5). CACM eligibility 
requirements are on a point system along with a written examination. Applicants must 
have a total of 7 points, with at least one point in each of three categories. The 
requirements for obtaining the CACM certification are detailed in Table 3. 

Unlike the CPCM, the CACM multiple-choice examination is based solely on the 
information contained in the Federal Acquisition Regulation (FAR) (NCMA, 2000b: 9). 
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The re-certification requirements for the CACM are identical to those of the CPCM. 
Each program allows lifetime certification for individuals at age 60 during their re¬ 
certification period, or retired from the profession at age 55 or older. 


Table 3. CACM Certification Requirements 


FORMAL 

EDUCATION 

ACQUISITION 
EDUCATION & 
TRAINING 

RELEVANT 

EXPERIENCE 

EXAMINATION 

Degree: 

Associates (2 pts) 
Bachelor’s (4 pts) 
Master’s/ 

Doctorate (6 pts) 

1 point for each 
course attended 
(courses must have a 
minimum 24 contact 
hours) 

1 point awarded for 
each year of 
experience 

Completion of CACM 
exam 

Semester Hrs: 

30-59 (1 pt) 

60-89 (2 pts) 
90-119(3 pts) 

120+(4 pts) 





Note. The formal education requirement may be waived if an applicant has 10 years experience 
and 24 CEUs, or has earned the Simplified Acquisition Specialist Credential and 10 CEU. 
Applicants must verify formal education, experience, and training requirements are met to be 
eligible for the examination. Table content derived from CACM Information Packet, Appendix 
A/1. 


National Association of Purchasing Management (NAPM). The scope of this study was 
limited to the evaluation of the two certification programs detailed above; however, in an 
effort to glean as much information as possible concerning the administration of 
certifications in general, all nationally recognized professional certifications for 
contracting were reviewed. Like the NCMA, the NAPM offers two distinct certifications 
to both members and non-members. NAPM, however, does recognize its Accredited 
Purchasing Practitioner (A.P.P.) certification as secondary to its Certified Purchasing 
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Manager (C.P.M.) program; individuals are not required to achieve the A.P.P certification 
prior to receiving the C.P.M. designation. 

NAPM was founded in 1915 with the stated purpose of providing “national and 
international leadership in purchasing and materials management, particularly in the areas 
of education, research and standards of excellence.” In 1974, NAPM established the 
C.P.M. program, designed for experienced individuals working in purchasing and supply 
management and focuses on managerial and leadership skills, a variety of specialized 
functions (e.g., sourcing analysis, supply and inventory management, forecasting), and 
operational aspects of the purchasing and supply function, such as identifying 
requirements, preparing solicitations and agreements, negotiations, technology, quality, 
and maintaining relationships. In 1996, NAPM supplemented its current certification 
processes through the establishment of the A.P.P. program for “(a) entry-level buyers 
who are primarily engaged in the tactical and operational side of purchasing, and (b) 
persons who work outside the organization's purchasing/supply management department, 
but nevertheless have definite procurement responsibilities.” (NAPM web site, 2001) 

The A.P.P. and C.P.M. are available to the defense acquisition workforce as well as 
those working in private and other not-for-profit business sectors. The requirements for 
achieving certification through NAPM are given below in Table 4. 

Both programs require re-certification every five years. The A.P.P. program calls for 
42 hours of continuing education through college courses (taken or taught), PCE (taken 
or taught), or the successful completion of any of the exam modules for either 
certification. C.P.M. certificants must re-certify by attaining 12 “points” through college 
courses (taken or taught), PCE (taken or taught), contributions to the profession, or the 
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successful completion of modules 3 and/or 4 of the latest C.P.M. examination. At least 
two-thirds of the points must be educational in nature. Both programs offer lifetime 
certifications for individuals documenting at least 18 years of full-time professional 
purchasing and supply management experience. (NAPM web site, 2001) 


Table 4. NAPM Certification Requirements 


CERTIFICATION 

LEVEL 

FORMAL 

EDUCATION 

EXPERIENCE 

EXAMINATION 

A.P.P. 

Associate’s Degree 

1 year professional 
experience 
(purchasing/supply 
mgmt may be a 
secondary duty) 

Successfully pass 
both modules of 
A.P.P. exam 

C.P.M 

Four-year Degree 

3 years full-time 
professional 
purchasing/supply 
mgmt experience 

Successfully pass all 
four modules of 
C.P.M. exam 


Note. The A.P.P. education requirement may be replaced by an additional year of professional 
work experience. The C.P.M. education requirement may be replaced with an additional two 
years of full-time purchasing and supply management experience. Table content derived from 
NAPM Official Web Site, 2001. 


National Institute of Governmental Purchasing (NIGP). The NIGP was founded in 1944 
and “provides its membership with education, research, technical assistance and 
networking opportunities in public purchasing; while promoting excellence, enhancing 
effectiveness and increasing public trust” (NIGP web site, 2001). NIGP initiated the 
Certified Public Purchasing Officer (CPPO) program, the first professional certification 
offered by a national professional purchasing association, in 1964, and the Professional 
Public Buyer (PPB) certificate in 1979 (UPPCC, 2000: 6). In 1978, NIGP joined with the 


18 




National Association of State Procurement Officials (NASPO) to establish the Universal 
Public Purchasing Certification Council (UPPCC) to “establish, monitor, and revise 
requirements for certification; to continue research efforts relating to the certification of 
public purchasers; to coordinate with other NIGP and NASPO programs in order to 
further the certification of public procurement officers; and to do all things necessary and 
proper to promote and insure professionalism in public procurement” (UPPCC, 2000: 6). 
The PPB certificate was re-named the Certified Professional Public Buyer (CPPB) in 
1991. 

In order to receive either certification, applicants must successfully complete the 
examination requirements respective to the program. In addition to the examination 
requirements, the UPPCC has established eligibility requirements that must be met in 
order to sit for an examination. The levels of professional education and experience vary 
with the level of formal education possessed by an applicant. The eligibility 
requirements for the CPPB and CPPO certificates are listed below in Tables 5 and 6. 

Once an applicant’s eligibility is verified, the applicant has 2 years to successfully 
complete the CPPB examination. The exam is comprised of 240 multiple choice and 
true-false questions in the areas of: Administrative Aspects of Purchasing, Procurement 
Requests, Solicitation and Evaluation of Bids/Proposals, Negotiation Process, Contract 
Award and Administration, Extemal/Intemal Relationships, Material Flow, and Inventory 
Management (UPPCC, 2000: 16). The UPPCC uses a point system to establish re¬ 
certification requirements. CPPB holders must re-certify every five years by attaining 10 
points in a minimum of two of the following three categories: (1) Membership; (2) 
Professional Contributions; and (3) Education and Training (UPPCC, 2000: 18). 
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Table 5. Certified Public Purchasing Buyer (CPPB) Eligibility Requirements 


LEVEL OF FORMAL 
EDUCATION 

PROFESSIONAL EDUCATION 

EXPERIENCE 

High School Diploma or 
GED 

12 college credit hours in 
purchasing or 96 contact hours 
in purchasing seminars 

5 years in purchasing; must 
include 3 years public 
purchasing 

Associate’s Degree 

Same as above 

4 years in purchasing; must 
include 3 years public 
purchasing 

Bachelor’s Degree 

6 college credit hours in 
purchasing or 48 contact hours 
in purchasing seminars 

2 years public purchasing 

Master’s or Advanced 
Degree 

3 college credit hours in 
purchasing or 24 contact hours 
in purchasing seminars 

2 years public purchasing 


Note. Table content derived from Universal Public Purchasing Certification Council Handbook, 
July 2000. 


CPPO applicants must complete a 240-question test similar to that administered to 
CPPB candidates but with a focus on buyer-related issues and an oral examination before 
a panel of CPPOs who evaluate the applicant’s communication and problem solving 
ability (UPPCC, 2000: 16). CPPOs must attain 15 re-certification points, in two of the 
three aforementioned categories, within each five-year re-certification period (UPPCC, 
2000: 18). CPPOs and CPPBs who have retired after at least 15 years of purchasing 
experience may apply for a lifetime certification at the time of retirement or at age 62 
(UPPCC, 2000: 19). 
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Table 6. Certified Public Purchasing Officer (CPPO) Eligibility Requirements 


LEVEL OF FORMAL 
EDUCATION 

PROFESSIONAL EDUCATION 

EXPERIENCE 

Holder of CPPB 
Certificate (less than 
Bachelor’s Degree) 

18 college credit hours in 
purchasing or 144 contact 
hours in purchasing seminars 

8 years in purchasing; must 
include 6 years public 
purchasing (3 of which 
must be in a management 
function) 

Bachelor’s Degree 

6 college credit hours in 
purchasing or 48 contact hours 
in purchasing seminars 

3 years in purchasing; must 
include 2 years public 
purchasing in a 
management function 

Master’s or Advanced 
Degree 

3 college credit hours in 
purchasing or 24 contact hours 
in purchasing seminars 

Same as Above 


Note. Table content derived from Universal Public Purchasing Certification Council Handbook, 
July 2000. 


Measured Qualifications for Certification 

The review of popular certification programs available to members of the contracting 
career field identified five measured qualifications used to determine an individual’s 
suitability for certification: (1) Formal Education; (2) Professional Education or Training; 
(3) Professional Experience; (4) Performance on a Comprehensive Examination; and (5) 
Continuing Professional Development (assessing the individual’s qualification for 
continued certification). In addition, all of the certification programs required certificants 
to uphold a standard of ethical behavior. Table 7 identifies the measured qualifications 
required under each certification program reviewed. 

Of all certifications reviewed, the Air Force’s APDP was the only program to certify 
individuals at progressive levels; other associations offer multiple certifications but do 
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Table 7. Certification Requirements Compared 


PROGRAM 

MEASURED QUALIFICATIONS REQUIRED 




Formal Education 

Experience 

Training Examination 

Re-Certification 

APDP (Air Force) 

X 

X 

X 



CACM (NCMA) 

X 

X 

X 

X 

X 

CPCM (NCMA) 

X 

X 

X 

X 

X 

A.P.P. (NAPM) 

X 

X 

X 

X 

X 

C.P.M. (NAPM) 

X 

X 

X 

X 

X 

CPPB (NIGP) 

X 

X 

X 

X 

X 

CPPO (NIGP) 

X 

X 

X 

X 

X 


not require one certification ahead of the other. The APDP was also the only program 
that did not require the completion of a comprehensive examination or periodic re¬ 
certification. The Air Force does have a continuous learning policy to ensure continued 
education and training; however, that policy is not currently enforced through the APDP 
certification process. The certifications offered through NCMA and NIGP differed from 
the others in that these programs required applicants to meet training, experience, and 
education requirements in order to be eligible to sit for the required examination. The 
APDP and NAPM certifications do not indicate an order for completing the various 
requirements. None of the certification programs reviewed required individuals to meet 
minimum education or experience requirements prior to attending professional training. 
Overall, the professional associations valued all categories of qualification as indicative 
of an individual’s suitability for certification, whereas the Air Force seems to have placed 
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confidence in the quality of its training to the extent that a comprehensive examination or 
re-certification are not necessary. 

The measured qualifications used to determine an individual’s suitability for 
certification seemed appropriate for indicating an individual’s professional status in that 
each supported at least one of Pavalko’s attributes of professions. Requiring formal 
education and professional training demonstrated the contracting career field demands a 
unique knowledge base. Requiring each certification applicant to have experience in the 
career field upheld the professional attributes of a long training period for indoctrination 
and a strong sense of identity as a separate profession. Preparing for a detailed 
examination and attending training courses require a significant investment of time and 
effort on the part of each potential certificant; these requirements supported the 
profession’s desire that individuals express a strong sense of commitment and loyalty to 
the profession. The requirement for continued development after initial certification 
reinforced the professional attributes of specialized knowledge, training, and commitment 
to the profession. Finally, as previously stated, each certification program required 
adherence to a code of ethics, linking professional certification to social values in 
addition to personal development; Pavalko found both of these features as qualities 
attributable to professions. 


Summary 

Based on a review of the relevant literature, the researcher developed some 
preliminary conclusions regarding the purposes for professional certification and the 
methods used by certifying associations to fulfill those purposes. 
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The DoD’s primary purpose in developing a mandatory certification process as well 
as encouraging the pursuit of other certifications was to prepare the acquisition workforce 
to implement continuing improvements to the defense acquisition process. Effectively, 
this purpose was accomplished through the professionalization of the workforce. 

Inherent in this method of overall process improvement, personal development and 
credential building for individuals assists in their career progression and advancement 
opportunities. The workforce includes commissioned officer, enlisted, and civilian 
personnel; although these three employee types must meet different requirements to enter 
the profession of defense acquisition, the researcher found all personnel to be subject to 
identical certification requirements. 

The certification programs available to members of the contracting career field, 
including the Air Force’s APDP, were reviewed and their requirements synopsized. This 
process of review identified five measured qualifications used to determine an 
individual’s suitability for certification. These qualifications were compared to Pavalko’s 
qualities of professions and were determined to serve the purpose of measuring an 
individual’s fitness for certification by establishing criteria to gauge the individual’s 
professional status in terms of the qualities of the overall profession. 

The next chapter will describe the methodology used to compare and evaluate the 
APDP and NCMA certification programs. 
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III. Methodology 


Overview 

This chapter describes the methodology employed to obtain the study’s data for 
analysis. Unlike most research methods, the techniques used in this study were not for 
the purpose of manipulating or analyzing existing data, but rather for actually creating 
and collecting original data. This chapter establishes the need for such a methodology 
and then reviews the principals for implementing that methodology. The chapter 
concludes by describing the research design. 

Basis 

The strategic goals of certifying professionals in the contracting career field are 
difficult to assess through measurement or statistical manipulation. To discover whether 
the current certification programs fulfill the purposes of certification discussed in Chapter 
II, it was necessary to look beyond the available hard data that record the volume of 
certifications granted or average examination scores. This study was more interested in 
whether certification of an individual actually represents and results in the betterment of 
the acquisition workforce as intended. In order to make such an assessment, it was 
necessary to rely on the soft data sources of judgment and opinion. 

While any individual with a measure of knowledge on the subject can offer opinion 
and enter discussion on the matter, this study required thought and judgment concerning 
strategic organizational objectives. This being the case, the discussion necessary to 
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generate the soft data required was conducted utilizing experts in the career field. The 
definition and selection of such individuals will be discussed in more detail later. 

Rather than obtaining individual expert assessments and comparing the results, 
experts had to carefully consider their positions in order to defend those positions in the 
face of alternative opinions held by other experts. When the best input on a topic is 
determined to be available from an expert, it logically follows that the input of multiple 
experts will achieve a superior result; however, the likelihood that experts will disagree 
requires the consolidation of inputs to form an overall “group opinion” (The Editors, 
1969: 80). Conducting such a discussion often takes place through the formation of 
Integrated Process Teams (IPTs) or round-table style sessions. The “group opinion” 
achieved through these methods can often be better described as compromise rather than 
consensus (The Editors, 1969: 80). An alternative to these traditional methods is an 
exchange format known as Delphi. 


Delphi 

Delphi has been defined as a “method for structuring a group communication process 
so that the process is effective in allowing a group of individuals, as a whole, to deal with 
a complex problem” (Linstone and Turhoff, 1975: 3). Delphi was developed in the early 
1950s during an Air Force sponsored study at The RAND Corporation to apply expert 
opinion in determining the number of A-bombs needed by the Soviets to reduce U.S. 
munitions output by a certain quantity (Dalkey and Helmer, 1963: 458). The method was 
initially believed to be primarily a prediction technique. Prior to 1974, the only known 
applications of Delphi within the Defense Department were for the purpose of forecasting 
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needs or outcomes (Elsbemd, 1974: 49-51); however in the years since its development, 
the method has been used for a variety of applications involving the evaluation of 
options, causal relationships, or any issue where the data requires a dimension of 
judgment (Linstone and Turhoff, 1975: 4). 

A review of the literature on Delphi revealed a lack of prescribed format for the 
conduct of studies under the method. Although there are no pre-established steps in 
performing Delphi experiments, there are basic principles and phases in the research. 
Dalkey and Helmer defined three distinct properties of any study naming Delphi as the 
method: (1) anonymity of panel members, (2) iteration and controlled feedback and (3) 
statistical group response (Dalkey, 1969: 16). The anonymity feature allows respondents 
to freely express opinions without the influence of intimidation or pressure to conform by 
dominant group members (Dalkey, 1969: 16). Dominant group members might be those 
recognized as the foremost expert on a subject or those of superior rank or position to 
other group members. If dominant members are permitted to influence the rest of the 
group it is likely that the discussion will not achieve results any better than those that 
could be attained by simply soliciting the single opinion of the most dominant group 
member. Controlled feedback is a method for eliminating extraneous material and 
consolidating duplicate responses by conducting iterations of discussion separated by a 
summary of the previous sessions inputs (Dalkey, 1969: 16). By formulating a statistical 
group response, the researcher assures each respondent’s input is present in the final 
response and reduces the pressure of group conformity by revealing the spread between 
varying opinions (Dalkey, 1969: 16). 
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Linstone and Turhoff find four basic phases of a Delphi discussion: (1) initial 
exploration of the issue, (2) formulating an understanding of group agreement or 
disagreement, (3) discovering and evaluating the underlying reasons for disagreement 
and (4) a feedback of the response evaluations to the group for consideration (Linstone 
and Turoff, 1975: 5-6). These four phases represent a single iteration of the process; 
multiple iterations will likely be necessary to achieve an accurate “group opinion”. The 
total number of iterations required will depend on the complexity of the issue and the 
amount of information available to the respondents. 

Since its development, Delphi has been subject to much testing and debate over the 
validity of the method. Several RAND studies initiated after the initial report by Dalkey 
and Helmer addressed the validity, value, and generalizability of the method (Dalkey, 
1969; Dalkey, Brown and Cochran, 1969,1970; Dalkey and Brown, 1971). A good 
synopsis of these studies is offered by Seaver, concluding that little experimental 
evidence is available to draw a definite conclusion when comparing the Delphi method 
with other methods of determining group judgment (Seaver, 1976: 42). Seaver also 
points to critiques by Pill (1971) and Sackman (1974) that attack the lack of scientific 
theory behind the method (Seaver, 1976: 43). Helmer admits as much in his foreword to 
Linstone and Turhoff stating, “Delphi still lacks a completely sound theoretical basis” 
(Linstone and Turhoff, 1975: xix). 

From the earliest application of Delphi, Dalkey and Helmer warned of dangers to the 
validity of a Delphi project. In the original Delphi study the experts interacted in daily 
operations and may have influenced one another’s response and the researchers used at 
least one respondent as a consultant (Dalkey et al., 1969: 466). The researchers also 
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admit that “leading” the respondents may be a result of the central synthesis of individual 
responses (Dalkey et al ., 1969: 467). These difficulties are not entirely unavoidable; 
however, the responsibility for protecting the study from such compromises must fall 
squarely on the researcher. 

Van de Ven and Delbecq also note that Delphi prevents the opportunity for social 
rewards and results in a feeling of detachment of group members from the problem, and 
that conflicts are not truly resolved where majority rule is employed as the determinant of 
group opinion (1974: 619). At the same time they concede that Delphi does offer a 
freedom from conformity pressure, results in a high quantity of idea input, and allows the 
respondents to thoroughly consider and formulate their positions (1974: 619). 

Though arguments against the theoretical basis of Delphi persist, there is no question 
that Delphi is an innovative technique for gathering expert judgments while controlling 
political dynamics and groupthink. Any researcher should thoroughly consider the 
applicability of Delphi to their method before embarking on a study. For the purposes of 
this research, Delphi is a facilitator for collecting a quantity of ideas from experts and a 
venue for discussion and argument. The principles of Delphi will be used to construct a 
research design that results in the receipt of the needed information; the methods of 
assessing the data achieved through Delphi will be statistical in nature and theoretically 
sound. This study will take the appropriate measures to ensure the discussion is directed 
in a manner that does not misrepresent any individual response and that the responses 
remain truly independent. 

This study benefited from the Delphi methodology most in that groups of experts 
were able to hold a detailed discussion over time with minimal interruption to daily 
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schedules so that participation could be heightened and result in a more comprehensive 
collection of ideas. Respondents did not have to travel, making the research significantly 
less costly. Since respondents were not confined to a single meeting to arrive at a group 
opinion, there was little pressure to conform in the interest of reaching an expedient 
conclusion. Finally, the anonymity of the respondents ensured each input was equally 
weighted and dominant personalities did not result in a distortion of the actual group 
opinion. 


Research Design 

Selecting Experts. Since no universal set of criteria exists for identifying an individual as 
an expert in a particular field, it was necessary to establish a set of criteria for the 
purposes of this study. The intent of the study was to gather opinions concerning the 
success or failure of current certification programs. It was imperative that participants 
had experience witnessing the effects of the certification programs on the acquisition 
workforce. To ensure this, participants were required to have a minimum 10 years 
experience in the career field and to have served in a supervisory capacity. To ensure 
each participant had first hand knowledge of the conduct of the certification programs, 
they were required to have successfully completed APDP Level III certification. A 
minimum pay-grade of 0-4 (military) / GS-12 (civilian) was established to demonstrate 
the participant’s expertise had resulted in successful career performance. In an effort to 
avoid a single organizational effect on the groups’ opinions, participants were selected 
from four MAJCOMs and included individuals working in both operational and systems 
contracting as well as headquarter administrative positions. 


30 




In determining the number of experts to include in the study, some guidance was 
found in literature recommending five to nine experts, urging a minimum of five (Meyer 
and Booker, 1991:87). Consistent with this recommendation, two separate groups were 
formed: one consisting of six active duty military members, the other consisting of six 
civilian employees. Following the terms of the Delphi methodology, the experts’ 
identities were known only to the researcher and that anonymity preserved throughout the 
conduct of discussions. A complete list of those participants granting the researcher 
permission to publish their identities in the final report is provided in Appendix A. 

Since all of the participants shared many similar characteristics in that all were 
APDP Level III certified, had comparable levels of experience, and filled a variety of 
duty positions, the researcher decided to use the individual’s status as either active duty 
military or civilian as the dividing point for the two groups. Forming two separate groups 
allowed the researcher to guide separate discussions, rendering the opportunity to explore 
multiple issues without overwhelming a single group. More importantly, it afforded the 
occasion to conduct a cross-case analysis of divergent responses and differing patterns 
between the groups. Such cross-case analysis has been determined to be important in 
guarding against premature conclusions and improving the probability for accurate and 
reliable deductions from the data (Eisenhardt, 1989: 541). 

Cycles of Discussion. Eight Delphi iterations were conducted during the course of the 
study. The iterations were divided into four separate cycles. The first cycle was 
comprised of a single iteration where participants submitted initial answers to five 
questions formulated to assess the value added by the measured certification 
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qualifications summarized in Chapter II: (1) Level of Formal Education; (2) Level of 
Professional Education or Training; (3) Level of Professional Experience; (4) Level of 
Performance on a Comprehensive Examination; and (5) Continuing Professional 
Development. In addition, the initial questionnaire included two questions concerning 
the appropriate number of certification levels and whether commissioned officers, 
enlisted, and civilian employees should meet the same requirements. In Chapter II, the 
researcher established that these measured qualifications served to operationalize several 
of Pavalko’s qualities attributable to professions; certification qualifications were focused 
on a specific purpose, but also supported strategic objectives. The panels were not 
instructed to answer the questions in terms of the qualities of professions, nor were they 
furnished with Pavalko’s list of attributes. The researcher used the first cycle of 
discussion to verify the credibility of the panelists as properly selected to assess the 
strategic goal fulfilling ability of certification programs. The responses to the initial 
questionnaire were analyzed to determine whether the experts were inclined to address 
professional certifications in strategic terms without prompting. In this manner the 
researcher validated two key principles of the study: (1) that professional certifications 
served strategic purposes; and (2) that the initial questionnaire was appropriate for 
generating discussion directed toward strategy. 

The second cycle consisted of five iterations, each readdressing a single item from 
the initial questionnaire. The participants were provided a synopsis of the comments 
generated during the first cycle and then asked additional questions allowing each to 
clarify or reevaluate his or her position in light of the input from other participants. 
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The third cycle asked each participant to review a synopsis of the discussion to that 
point and then select a position on the optimal number of certification levels; a single 
iteration was used. The participants then commented on the format of the other measured 
certification qualifications within the bias of their position on certification levels. 

The fourth cycle used the final iteration to ask for each participant’s 
recommendations for changes to the current certification programs, based on the 
discussion conducted in the previous cycles. 

Assessing the Certification Programs. Following the completion of the discussion 
cycles, the researcher gathered all expert input and attempted to glean an overall group 
opinion. Comparing the group discussion data to existing certification requirements, the 
researcher then made recommendations that the experts believe will improve the current 
certification programs and benefit the professional development of the Air Force 
contracting workforce. The findings of the study follow in Chapter IV. 
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IV. Findings and Analysis 


Overview 

This chapter provides a detailed account of the results of the research accomplished 
under the design and methodology described in Chapter III. Prior to the discussion 
cycles, the researcher provided each participant an overview of the study and a 
Participant Profile Questionnaire. This initial information can be found in Appendix B. 
The study included eight iterations of discussion within four distinct Delphi cycles. The 
results of the discussion are presented in chronological order by cycle. 

Delphi Cycle 1 

The first cycle of discussion included the dissemination and subsequent response to 
an initial questionnaire. The response rate to the initial questionnaire was 100 percent. 
Participants were asked seven questions concerning the certification of contracting 
professionals in general. Since this was the first cycle of discussion, both the military 
and civilian panel received the same questionnaire; however, the responses were analyzed 
separately so that subsequent cycles of discussion could be tailored specifically to the 
group. The questionnaire is presented in full text in Appendix C; a synopsis of specific 
responses is provided in Appendix D. 
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Question 1. What value does formal education provide to the workforce that cannot be 
achieved through experience and training? What specific degrees or disciplines should 
be required? At what level (BS, MS, PhD...)? 

Civilian Responses. All respondents believed that formal education provided value 
to the workforce. Panel members stated that formal education: “supplies a broad base of 
learning, well-rounded personnel; draws on topics not focused on in training or achieved 
through experience,” “adds credibility and credentials,” and “facilitates a network that 
can be utilized in the work world.” The group also agreed that degree requirements need 
not be limited to a business curriculum; that the process of formal education in any 
discipline improves the workforce. Overall, respondents were in favor of formal 
education as a requirement for certification, although there was some disagreement on the 
level of education to require. 

There were different thoughts on the level of education that should be required. 

Some respondents recommended different requirements for different levels of 
certification, others concentrated on an individual’s position or grade level. Overall, it 
was not clear, at the end of the first cycle, whether the group believed the formal 
education requirement must be met through the completion of a degree or a lesser amount 
of coursework. Some respondents recommended different requirements for enlisted 
personnel, others made no such distinction. Two respondents did not mention a 
particular level of education to require. In an effort to clarify the overall group opinion in 
this area, the subject was reevaluated through a different but related line of questioning in 
Cycle 2a. 
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Military Responses. All respondents believed that formal education provided value 
to the workforce. As with the responses from the civilian panel, the group emphasized 
the academic process and the type of thinking it requires establishes a valuable 
foundation for contracting professionals: “formal education exposes individuals to a 
wider variety of topics and learning situations than normally possible though experience 
in a work environment,” “individuals completing degrees prove their ability to see a long¬ 
term task through to completion,” and . .it develops an individual’s ability to critically 
analyze and apply information that is produced by others.” Unlike the civilian panel, four 
of the six respondents agreed that the formal education should be in business or business 
related areas. Overall, respondents were in favor of formal education as a requirement 
for certification, although there was some disagreement on the level of education to 
require. 

The group’s opinion concerning the appropriate level of education to require was not 
clearly definable at the end of Cycle 1. One respondent recommended: “to enter the 
career field it should be a BS.” Another would only require a degree “in order to attain 
top certification.” One panel member specifically noted that enlisted personnel have 
successfully performed duties in the career field without a degree requirement. Few of 
the respondents actually spoke of formal education in terms of the professional 
certification process. The researcher continued the discussion on the subject in Cycle 2a. 

Question 2. The training requirement of most certification programs is accomplished 
through the completion of PCE courses. What value, if any, does this type of training 
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offer beyond that received at the workplace? What specific courses or subject areas 
should be required? 

Civilian Responses. All respondents believed that Professional Continuing 
Education (PCE) classroom training provides value to the workforce. One respondent 
stated, “PCE classes expose our employees to a range of problems that might not be 
encountered on a regular basis in the work place.” Another panel member commented on 
the necessity for employees to stay current, “CPE courses are the best way of getting new 
ideas and procedures out to the field personnel.” Two respondents found value in the 
networking opportunities afforded by in-residence training with other workforce 
members. Overall, the group agreed that PCE plays an important role in professional 
development and the improvement of the workforce. 

While the entire group found value in requiring individuals to attend PCE courses, 
there was some level of disagreement as to the specific courses to require for 
certification. Three respondents did not mention specific courses or subject areas to 
require. Some additional issues were also raised, including the congruity between 
certification requirements and the training needs of the workforce, “prior to the 
APDP.. .virtually 100% of our workforce had completed more than the requisite 
courses.. .certification requirements have had little to no impact on the PCE training our 
workforce receives,” and the experience level that should be required prior to attending 
PCE, “if [PCE] is received before on-the-job training, the individual may have difficulty 
in finding the correct frame of reference for the topics being taught.” In response to these 
areas requiring further discussion, the researcher developed follow up questions 
administered in Cycle 2b. 


37 




Military Responses. All respondents found some value in Professional Continuing 
Education (PCE). As with the civilian panel, the group attributed this finding to the 
controlled training environment and especially to the value added through establishing a 
communication network with other members of the Air Force’s acquisition workforce: 
“the best environment for exposure to numerous situations and decisions in a short time is 
the classroom,” . .courses expose an individual to contracting methods and ideas which 
they may not get exposed to during OJT,” and “I think we’re tending to computer based 
training...that’s a shame, it eliminates the interaction with classmates, it destroys the 
sense of community that allows us to call each other and ask questions comfortably.” 
Overall, the group was in favor of PCE as a certification requirement. 

Only three of the six panel members indicated specific coursework to require for 
certification. Two of those three supported the current curriculum required under the 
APDP certification program, the other listed specific courses. The researcher could not 
determine an overall group opinion for that portion of the question at the end of Cycle 1. 
Similar to the responses of the civilian panel, one group member questioned whether 
certification requirements actually result in a higher level of training through PCE. The 
discussion regarding PCE was resumed in Cycle 2b. 

Question 3. What is the value of requiring a minimum amount of contracting_experience 
before granting certification? What is the appropriate way to measure that experience? 
What specific tasks or areas of experience should be required? 

Civilian Responses. Five of the six panel members commented on the value of 
experience. While the remaining member did not speak of the general value of work 
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experience, the member’s response did indicate a preference for an experience 
requirement in the certification programs. Overall, the group believed that experience is 
essential for determining an individual’s level of professional development. One 
respondent described the value of experience by saying, “education [and] training are 
only effective when put into practice,” while another panelist drew a distinction between 
training and experience: 

Discussing the problem in an academic setting usually does not involve the 
emotional problems that are inherent in real world settings.. .experience gives 
the professional the ability to recognize those areas where emotional issues 
might be getting in the way of a needed solution...there is seldom a clear cut text 
book solution. 

The group tied experience to the other certification requirements, claiming that academic 
and PCE coursework will only benefit the individual and the workforce when the skills 
presented in those settings are applied to real situations. 

Four of the six respondents submitted a particular method of measuring experience 
before granting certification; two experts agreed with the current method using time in 
the career field, another recommended specific experience in different duty areas be 
required. The fourth expert responding to this portion of the question suggested, “tying 
the PCO/ACO warrant (unlimited variety) as a pre-requisite to certification.” This 
innovative perspective would be addressed to the entire group in a later cycle. 

At the conclusion of Cycle 1, it was not possible to determine the overall group 
opinion of the best measure of experience. The measurement used by certification 
programs at the time of discussion was the number of years of experience possessed by 
an applicant. There was some question among the group members as to the effectiveness 
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of this measurement in ensuring an appropriate level of experience. This issue was 
readdressed during Cycle 2c. 

Military Responses. Following the first cycle of discussion, the researcher was able 
to determine that the entire group valued experience as a component of the certification 
process. One response captured the essence of the group opinion: “there is no substitute 
for experience.. .no amount of education will be sufficient unless you can include 
practical, hands-on contracting experience.” There was also a strong indication that the 
group preferred a varied base of experience to specialization in one aspect of acquisition. 
Four of the six respondents stated that experience should be measured in a more task 
specific manner than was the case with the current method; one recommended a current 
method used outside the certification process: “the enlisted OJT skills upgrade is the best 
way I’ve seen to measure experience.. .a list of core tasks that any contracting 
professional would need to know.” Those respondents agreeing with the current measure 
of number of years experience admitted it was not the optimal measure for the 
requirement but saw no viable alternative. Overall, the group agreed the current measure 
of experience was inadequate. 

Only two of the respondents listed specific areas of experience to require. In either 
case, the respondents did not offer a rationale in support of their statements, only 
examples of their overall view of experience requirements. The researcher developed 
additional questions regarding the level of experience to require and improvements to the 
existing certification requirements for experience and continued the discussion during 
Cycle 2c. 
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Question 4. What is the value added by requiring the successful completion of a 
comprehensive examination before granting certification? 

Civilian Responses. Of six respondents, four were in favor of an examination 
requirement while two were not. Proponents of such a requirement argued that 
successfully completing an examination requires the individual to “[demonstrate] they 
have a specified level of mastery of the material,” and “study areas which they would not 
normally deal with on an everyday basis.” However, the four respondents favoring an 
examination requirement did not universally agree that the examination should be 
required at all levels of certification. Dissenting opinions expressed that “we have study 
programs which tend to teach the test.. .not optimal,” and “if formal education, training, 
and experience are required.. .only consider examinations to substitute or waive formal 
education.” 

With only a 75 percent rate of agreement on the issue, and considering some 
differences between affirmative responses, the researcher determined there would be 
benefit from further discussion. The issue was resubmitted to the group during Cycle 2d. 

Military Responses. The responses to this question varied widely in Cycle 1. The 
researcher could not determine the group opinion on any level. The panel members were 
provided the Cycle 1 responses and asked to reevaluate the subject during Cycle 2d. 

Question 5. What value is provided to the workforce by requiring re-certification? 

Which categories of development should be included in the re-certification requirement? 


41 



Civilian Responses. All of the panel members agreed that continuing training and 
education after certification is valuable to the profession. Overall, the group believed 
individuals should stay current in the practices they employ to perform their duties. 

Although the group found value in continued education and training, not all members 
agreed that a formal reestablishment of an individual’s certification should be required. 
The DoD implemented its Continuous Learning policy, but does not revoke certifications. 
Some group members believe this policy to be adequate for ensuring continued 
development, finding no value in re-certification “as long as on-going training is 
required,” or “if a graduated/progressive certification approach is in place.” One panelist 
believed the re-certification requirements was of more value “to the sponsoring 
association than to the individual.” Those seeking to require re-certification would 
concentrate the requirement around continuing education in areas subject to the most 
significant change. 

Military Responses. Four of the six panel members favored re-certification in that it 
requires the individual to continue their training and education. The agreeing panel 
members cited that “contracting is changing constantly,” and “[the] requirement ensures 
the employee continues to be exposed to current trends.” The expectation that 
individual’s should demonstrate an ongoing commitment to the profession also factored 
into the preference for re-certification. 

Two of the panel members expressed a disagreement with any requirement for re¬ 
certification: “absent a comprehensive examination I would not require re-certification,” 
and “seemed like a method to get me to send [more] money and demonstrated no value 
added.” These responses, however, did not directly argue against any of the points made 
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by the agreeing panel members. The researcher did not dedicate a single iteration to 
redirecting the arguments concerning re-certification; however, suspecting the two 
disagreeing members would favor some level of continued education and training, the 
issue was approached again in Cycle 3. The dissenting opinions were synopsized and 
offered to a combined military/civilian group at that time. 

Question 6. What is the value of certifying professionals at progressive levels, as 
opposed to providing a single certification for all personnel? 

Civilian Responses. Four of the six panel members found some value in a 
progressive certification program. They believed that a certification at an appropriate 
level provides an indicator of the individual’s actual abilities. The group commented that 
progressive levels help ensure only those qualified received the highest certification. 

Two panel members agreed that progressive levels offer interim incentives to individuals 
as they strive to achieve the highest level of development. 

Since the group agreed that training and education should be ongoing throughout an 
individual’s career, it seemed likely that the group would agree that certification should 
be an ongoing experience. However, two panel members did not find value in a 
certification at progressive levels, one stated that lower levels of certification were “more 
a mark against [the individual].. .kind of like a consolation prize.. .it says, ‘not good 
enough for the first team.’ ” The researcher determined there to be a sufficient level of 
disagreement to warrant further discussion. Using progressive levels within a 
certification program was reevaluated during Cycle 2e. 
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Military Responses. Five of the six panel members stated a desire to certify 
individuals at progressive levels. The remaining respondent indicated uncertainty as to 
the definition of “progressive levels” and did not indicate a position during Cycle 1. 

Those in support of progressive levels indicated that “intermediate goals are important for 
professional development,” and also that “[graduated certification] is a better indication 
of an individual’s level of ability, experience, and responsibility,” offering a check and 
balance system between time in the career field and the expected level of certification. 

The level of agreement on this issue was at a point that led the researcher to 
determine a second iteration dedicated to the question concerning progressive levels was 
unnecessary. The panel members were given the opportunity to verify their position on 
progressive levels during Cycles 3 and 4. 

Question 7. Please review your answers to the questions above. Should there be 
different requirements for military and government civilian personnel? For DoD and 
private industry personnel? 

Civilian Responses. Five of the six panel members stated that military and civilian 
members of the acquisition workforce should meet the same requirements for 
certification. Military and civilian members often execute very similar duties and require 
the same level of professional development. One expert designated an exemption for 
enlisted members saying, “it will be very difficult to apply all the above criteria to 
enlisted personnel.. .few of them could meet all the qualifications.” Although not 
specifically stated, it is presumed the education requirements would be the most difficult 
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for enlisted to meet. The group agreed that all acquisition professionals should pursue 
certification. 

Although the group opinion regarding the certification requirements for military and 
civilian acquisition professionals was near unanimous, some other related issues did arise 
from the Cycle 1 responses. Two panel members distinguished enlisted contracting 
professionals involved in contingency operations as needing different or additional 
certification requirements. Other responses offered differing opinions regarding the 
similarities between DoD and private industry certification needs. In order to effectively 
discuss possible differences between the needs of enlisted members and officers, or 
between defense acquisition and private industry, it would be necessary to solicit 
responses from experts in those specific areas of the acquisition community. These 
issues were not anticipated at the outset of the study and the panels established did not 
meet those profiles. While these issues are valid and merit further discussion, it was 
determined they were outside the scope of this particular research design. Question 7 
was not discussed further in subsequent cycles. 

Military Responses. Similar to the civilian panel, five of the six group members 
stated that military and civilian members of the acquisition workforce should meet the 
same requirements for certification. One respondent made the point that civilians even 
perform contingency contracting duties, especially when the contingency does not require 
deployment. The disagreeing panel member believed that “military should be required to 
attend more contracting schools, in-residence, which deal with contracting in deployed 
locations.” 
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Subsequent to the first cycle, it was determined that further discussion concerning 
Question 7 was outside the scope of this particular study. The military panel touched on 
some of the same issues presented by the civilian group. A recommendation for 
additional research will be offered in Chapter V. 

Cross-case Comparison. At the conclusion of the first cycle, the discussions within the 
separate groups had taken on similar characteristics. Overall, the panel members did not 
limit their comments to those applicable to their particular employment status; civilians 
often spoke of the functions that military members perform and vice-versa. Both groups 
made some mention of the enlisted forces. Where unique ideas were offered within a 
group, a similar response was noted from a member of the other group. For example, 
both civilian and military panel members commented that re-certification was of more 
value to the certifying association than to the workforce. One isolated difference was that 
only the military panel commented on the difference between CBT and in-residence 
training. However, this concept was introduced by a single panel member and could not 
be determined to represent an overall difference in view between the military and civilian 
panels. Although specific comments varied between groups, there was like variance in 
the comments within each group. At the end of Cycle 1, the researcher did not discern 
any divergence in the discussion between groups that could be attributed as a function of 
group composition. 

Each group validated the assumption that professional certification programs served 
strategic purposes. Although the panelists were not provided with a list of Pavalko’s 
qualities attributable to professions, the experts repeatedly expressed the value of 
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certification elements in these terms. The panelists found that individuals should possess 
a significant level of specialized knowledge to claim expertise in the career field; the 
experts sought certification requirements that would ensure an individual’s mastery of 
contracting processes. Several panel members expressed a desire for more stringent 
experience requirements to ensure adequate indoctrination to the profession prior to 
certifying an individual as a professional. The group members also mentioned the social 
responsibility of defense acquisition personnel expending public funds and the need for 
qualified individuals to ensure that responsibility is properly carried out. A number of 
experts cited the importance of networking, supporting that contracting is a significant 
subculture within the acquisition community. Overall, the groups did not suggest that 
professional certification was necessary only to meet the requirements of DAWLA, the 
experts found that certification benefits the workforce and profession as a whole in 
addition to the individual benefits received by certificants. 

Delphi Cycle 2 

The second cycle of discussion was comprised of five Delphi iterations. Each 
iteration addressed an area determined to require further discussion in Cycle 1. Again, 
discussions were conducted separately between the two expert panels. Unlike the first 
cycle, the questions submitted to the two panels were not necessarily identical; questions 
were tailored to the panels’ remarks during Cycle 1. 

Civilian Responses for Cycle 2a. The response rate for this iteration was 100 percent. 

The participants were asked four questions concerning professional certification 
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requirements for formal education. The questionnaire is presented in full text in 
Appendix E; a synopsis of specific responses is provided in Appendix G. 

Question 1. Why is it important for the individual to possess an undergraduate 
degree before being granted entrance into the contracting career field? 

Three of the six panel members did not believe an undergraduate degree should be 
required to enter the career field. Although the respondents did not offer support for this 
position, each went on to say that a degree should be required before any individual is 
allowed to advance to the grade of GS-11 or above. The remaining three panel members 
were in favor of requiring a degree for entrance in the career field. These respondents 
stated, “we have been on a long path to professionalize the contracting career field...the 
degree requirement is one method to take us farther on that journey,” and “a degree is 
necessary just to establish a level playing field [between government personnel and their 
private sector counterparts].” The experts also expressed that a degree demonstrates a 
level of commitment and work ethic and increases the credibility of the career field 
within the acquisition community. One respondent referred to the National Defense 
Authorization Act for 2001 that made changes to the qualification requirements for 
contracting personnel. Public Law 106-398 (Section 808), amended Section 1724 of title 
10, United States Code to require civilian employees and military members of similar 
occupational specialties to possess a degree, with a major or concentration in business, 
before they are granted entry into the career field, or before they move to a new position 
if they are already in the career field. This response played a significant role in guiding 
the discussion during Cycle 3. Overall, the expert panel valued formal education and 
agreed it should be a requirement, at some level, for all contracting professionals. 
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Question 2. Considering APDP and other contracting professional certifications 

(CPCM, CACM...), why is it important for the individual to possess an undergraduate 

degree before being granted professional certification? 

Each respondent made reference to the term “professional” or “professionalism” in 

answering the question. The panel members were in complete agreement that a 

professional certification should require individuals to obtain degrees. The experts stated 

that a certification program with rigorous requirements provides more value and 

credibility to those who are certified. Several of the respondents made strong statements 

challenging any professional certification not requiring the completion of a degree 

program. One respondent went as far to suggest that the APDP certifications are not truly 

professional certifications as are those offered by other associations that require degrees: 

I think it is important to differentiate between the APDP “box top” certifications 
(just send in your box tops in the form of completed courses and you are 
certified for life) and the NCMA professional certifications.. .An undergraduate 
degree is a reasonable prerequisite for certification in a quasi-professional field 
such as contracting. 

Question 3. Under what circumstances would a formal education requirement be 
waived? 

The panel offered very limited exceptions to requiring formal education prior to 
certification. Two respondents stated they would waive the requirement for individuals 
possessing at least 10 years of experience; one required managerial experience, the other 
required the 10 years be completed prior to October 1991 (a current APDP waiver). 
Another panel member stated an exemption for enlisted members in contracting. Enlisted 
personnel are not required to receive a degree before entering the service and are vital to 
the Air Force’s contingency contracting mission. A fourth panel member found reason to 
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exempt those first entering the program, but qualified the waiver by requiring those 
individuals to be actively pursuing an undergraduate degree. Two panel members 
refused any waiver. Overall, the group supported their preference for formal education 
requirements by placing extremely stringent restrictions on waivers. 

Question 4. Does your answer to Question 1 apply equally to officer, enlisted, and 
civilian personnel? Why or why not? 

All six-panel members agreed that formal education requirements should be the same 
for active duty officers and civilians in the contracting career field. The group believed 
that professional certifications should be a consistent indicator of professional 
development for all certificate holders, regardless of the particular position they occupy. 
Three of the panel members went on to suggest that enlisted members should also be held 
to the same requirements for certification. Although enlisted members are not required to 
receive formal education as a condition of employment, these experts recommended they 
be expected to fulfill the same certification requirements as officers and civilians: “I 
envision a merit based career field wherein individuals are judged by their knowledge 
and accomplishments, not by the method they entered the workforce,” “the level of 
certification demonstrates a level of professional development, not position held,” and 
. .the Air Force is going to have to change the thinking on enlisted Contingency 
Contracting Officers...” The remaining panel members did allow enlisted a reprieve for 
fomial education requirements, but did not specifically state whether the requirements for 
career field entrance or certification, or both, should be relaxed for enlisted troops. 
Overall, the panel was of the opinion that formal education requirements are important to 
the certification process and should be applied consistently to all contracting members, 
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with a possible exception for enlisted members, in order to maintain the prestige and 
credibility of the certification. 

Military Responses for Cycle 2a. The response rate for this iteration was 84 percent (5 of 
6 panel members responded). The participants were asked four questions concerning 
professional certification requirements for formal education. The fourth question differed 
from that asked of the civilian panel. The questionnaire is presented in full text in 
Appendix F; a synopsis of specific responses is provided in Appendix G. 

Question 1. Wiry is it important for the individual to possess an undergraduate 
degree before being granted entrance into the contracting career field? 

Two of the five respondents did not call for an undergraduate degree as an absolute 
requirement for entrance. One of those panel members did recommend the degree 
requirement as a significant factor for determining advancement within the career field. 
The other expert stated that the Air Force would find it “very difficult to meet the mission 
currently fulfilled by the enlisted force” if a degree were required of those individuals. 
This panel member also believed the Air Force’s ability to provide contracting support to 
operational wings could be hampered if civilian employees at all levels were required to 
obtain degrees. The remaining three respondents believed degrees should be required for 
entrance; however, one of these experts agreed with the exception for enlisted members. 
Those supporting a degree entrance requirement stated that the role of the contracting 
specialist is becoming that of a “business advisor” and that formal education in that area 
was imperative. One respondent also pointed to the requirements in other professions 
such as medical, legal, and accounting, “Why would we ever consider anything less for 
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our profession?” suggesting that a degree entrance requirement is necessary for the career 
field to be considered a true profession. Overall, no panel member disputed the 
importance of formal education and most supported requiring an undergraduate degree 
for entrance into the career field. The only exceptions to this opinion were expressed as a 
concern for mission continuity rather than in dispute of the importance of formal 
education. 

Question 2. Why is it important for the individual to possess an undergraduate 
degree before being granted professional certification? 

Like the civilian panel, the military contracting experts believed that certification 
should be reserved for those demonstrating superior capability. The panel members 
referred to an undergraduate degree requirement as essential to a certification program 
seeking to be designated as a professional certification: “a certification should be built on 
the degree.. .should show a person has gone through a stringent program to obtain 
certification.” One panel member did offer an exception for the Air Force’s APDP 
certification. This expert would not require Level I certificants to complete an 
undergraduate degree but suggested a bachelor’s degree be required for Level II and a 
graduate degree for Level III, providing the anticipated effects on the workforce were 
favorable. Overall, the panel agreed that formal education should be required for 
certification; all but one respondent believed that requirement should insist on the 
completion of an undergraduate degree. 
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Question 3. Under what circumstances would a formal education requirement be 
waived? 

All respondents expressed an exception to the degree requirement; however, the 
panel did not agree on the circumstances for such an exemption. Two experts referred to 
the enlisted contracting forces as eligible for a waiver for the same reasons they cited in 
Questions 1 and 2. Another pair of experts agreed that the requirement might be waived 
if the individual had extensive experience in contracting; however, neither panel member 
expressed this exception adamantly, one stated the waiver should only be for “a period 
long enough to give the individual time to finish their formal education.” The remaining 
panel member did not require a degree for entrance or certification and did not comment 
on the waiverability of a lesser education requirement. Overall, the expert panel 
recommended only limited exceptions to formal education requirements. 

Question 4. What value does formal education in a business related discipline 
provide to the workforce that cannot be achieved through other means (e.g., formal 
education in other disciplines, experience or training)? 

Three of the five respondents believed that a business degree offered value beyond 
that of other disciplines. These experts stated that contracting professionals must 
understand business operations to effectively communicate, negotiate, and partner with 
contractors, and that a business degree also established leadership and management skills. 
One panel member conceded that other disciplines can render specific business 
knowledge effectively, but held that a business degree gives the individual “a breadth of 
knowledge in all the different disciplines [logistics, acquisition, manufacturing, 
accounting, etc].” Two of the other experts did not find the need for a complete degree in 
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business; one stated that 24 credit hours of business coursework could meet the objective; 
the other believed any accredited program would establish the academic foundation 
needed. One of the previously mentioned panel members disputed this assertion stating 
that certain undergraduate programs (psychology and sociology) had lower standards 
than a good business degree, “these degrees are a joke.. .for football players to major in.” 
The remaining panel member did not find a business degree more valuable than other 
disciplines. Overall, the group agreed certain knowledge and skills are gained through 
formal education and most panel members believed a significant portion of this formal 
education should be in business-related coursework. 

Civilian Responses for Cycle 2b. The response rate for this iteration was 84 percent (5 of 
6 panel members responded). The participants were asked four questions concerning 
professional continuing education training requirements for certification. The researcher 
provided the panel a synopsis of the comments submitted during Cycle 1. The first 
question was designed to verify the apparent group agreement following the first cycle; 
the remaining three questions were developed based on issues raised by the panel during 
the first cycle. The questionnaire is presented in full text in Appendix H; a synopsis of 
specific responses is provided in Appendix J. 

Question 1. To what extent do you agree with requiring the courses listed above 
[Contract Law, Contract Administration, Pricing, Negotiation Techniques, Market 
Research, and Quantitative Methods]? 

Three of the five respondents agreed with including all of the courses listed in a 
contracting certification program; one panel member did not expressly agree nor disagree 
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with a particular course. The remaining expert stated that a separate Market Research 
course should not be required, but that current course curriculums should increase their 
emphasis in that area. Overall, the panel agreed with the courses recommended during 
Cycle 1; the experts were again given the opportunity to suggest changes to current 
course offerings in Cycle 4. 

Question 2. Do the current certification programs satisfy these requirements? 

The panel did not believe the current certification requirements satisfied the training 
needs of the workforce; one expert responded affirmatively but also agreed with requiring 
Quantitative Methods, a course not present in current requirements. The group believed 
the workforce would benefit from broader training accomplished through additional 
courses such as Quantitative Methods and Cost Accounting, or by initiating a 
requirement for an elective course from a field other than contracting. One expert added 
that the current curriculum does not sufficiently develop essential interpersonal and 
communication skills: “negotiating skills, meeting skills, personal interaction, and 
organizational behavior are very important.. .should be mandatory for anyone at the GS- 
12 level and above.” As previously stated, one panel member thought market research 
subject material should be enhanced throughout the curriculum. Overall, the panel 
accepted that current requirements meet the most basic needs of the workforce, but 
recommended a stronger curriculum be developed. 

Question 3. To what extent has establishing certification programs emphasized PCE 
for the workforce? Is this favorable? 

The group agreed that the development of certification programs (namely, the 
mandatory APDP certifications) have had a favorable effect on the emphasis of PCE for 
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the workforce. One panel member stated, “APDP makes for a much more orderly 
acquisition of contracting knowledge than was previously possible,” another added, “it 
forced management to insure everyone in the field received training.. .it has been very 
favorable.” However, two panelists did comment that the overall effect, while favorable, 
has been less than significant. One expert cited the lack of a re-certification requirement 
in APDP failed to motivate continued training beyond the minimum requirements. 
Overall, the panel concluded that certification programs do emphasize PCE for the 
workforce and that emphasis generates favorable results, but that the extent that the 
APDP certification program strengthens the training of the workforce could stand to be 
increased. 

Question 4. What experience should be required before attending PCE? 

The experts agreed that some exposure to the work environment should be required 
before individuals begin attending PCE training. The precise amount of experience to 
require varied between the members; panel members advised as few as three months and 
as many as eight months experience before one attends their first PCE course. Current 
certification requirements insist on a minimum level of experience before certification 
but do not necessarily require a specific amount of experience before completing the PCE 
requirements for certification. Theoretically, an individual could complete all training 
requirements without ever performing contracting duties and then gain certification after 
the minimum required time in the career field; of course, a portion of the experience 
requirement would be achieved during the time spent attending training, rather than 
gaining actual work experience. Overall, the group believed it was unadvisable to 
expedite or compress the classroom-training schedule for an individual so that they were 
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certified with minimum experience. This practice may result in unqualified certificants 
and detract from the credibility of the certification. 

Military Responses for Cycle 2b. The response rate for this iteration was 67 percent (4 of 
6 panel members responded). The participants were asked four questions concerning 
professional continuing education training requirements for certification. The researcher 
provided the panel a synopsis of the comments submitted during Cycle 1. The first 
question was designed to verify the apparent group agreement following the first cycle; 
the remaining three questions were developed based on issues raised by the panel during 
the first cycle. Based on the responses to Cycle 1 questions, Questions 3 and 4 asked of 
the military panel differed from those submitted to the civilian experts. The 
questionnaire is presented in full text in Appendix I; a synopsis of specific responses is 
provided in Appendix J. 

Question 1. To what extent do you agree with requiring the courses listed above 
[Source Selection/Best Value, Contract Management, Contract Law, Services, 
Construction, Systems and Contingency Contracting, Pricing, and Electronic 
Contracting]? 

The experts agreed with all of the courses listed. The panel believed that 
certification should be a demanding process and that only those with a high level of 
knowledge and training should be certified. One panelist stated that these topics envelop 
the actual situations trainees will face in the workplace and that discussion and 
instruction of these issues in a classroom setting “prepares the student for hundreds of 
real world events that they would not experience otherwise.” The panel agreed with 
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specialized training in electronic systems, especially with the impending 
operationalization of the Standard Procurement System (SPS). One panel member also 
recommended coursework in commercial contracting. 

Question 2. Do the current certification programs satisfy these requirements? 

Overall, the group was not satisfied with current certification requirements. Two 
panelists stated that the curriculum should be more rigorous in order to provide adequate 
training in all the required areas; these experts believed the current offerings to be 
“square fillers”. One member remarked that “we had a better trained workforce before 
the certification programs, our problem was that it wasn’t evenly spread”; requiring 
certification has guaranteed widespread training but that the level of training offered has 
diminished to accommodate the demand. Another panelist did not think the certification 
process guaranteed relevant training; individuals will be sent to required courses for 
certification ahead of other courses that may be more immediately applicable to their 
duties. 

Question 3. How is the workforce benefiting from the level of difficulty in current 
course offerings? 

The respondents verified their assertion that the current coursework is too facile and 
that the benefit of certification is subsequently diminished. In the experience of these 
experts, all attendees pass required courses with very little effort, one stated, “they all 
pass if they show up and breathe.” One respondent remarked that there is an unfavorable 
career connotation for those who leave the field for an extended period to fill instructor 
positions; therefore, it is “tough to get quality folks teaching our contracting 
professionals.” Course material is standardized at the AETC level and instructors are not 
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given the freedom to tailor the level of difficulty to the abilities of the students. Overall, 
the group found benefit in PCE training for the workforce but did not believe the 
potential benefit is realized through the mandatory APDP certification process. 

Question 4. Why should certification programs require in-residence PCE, rather than 
giving credit for on-line or software instruction? 

The respondents unanimously favored in-residence training to that received on-line. 
One panel member stated “basic courses could be taken on-line but that higher-level 
courses should be case study oriented” and require critical thinking and free flowing 
discussion. The remaining panelists insisted that classroom interaction is invaluable to 
the learning process. The opportunity for students to “learn from each other” and 
continue communication when they return to work was seen as a priority in the training 
process. The group believed that in-residence interaction is the source of much of the 
benefit from training, simply completing reading and examinations are only successful in 
meeting minimum requirements. One of these experts recommended requiring Computer 
Based Training (CBT) as a prerequisite to in-residence attendance in order to “increase 
the quality of the interaction.” Based on the answers to Questions 3 and 4, it was 
apparent the group considers the current mandatory certification program places greater 
importance on the completion of training than the quality of training. 

Civilian Responses to Cycle 2c. The response for this iteration was 67 percent (4 of 6 
panel members responded). The expert panel was provided a synopsis of Cycle 1 
responses as well as the current APDP and CPCM certification requirements for 
experience. The researcher submitted two additional questions concerning requirements 
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for experience in contracting professional certifications. The questionnaire is presented 
in full text in Appendix K; a synopsis of specific responses is provided in Appendix M. 

Question 1. Should the experience requirements insist on experience in more than 
one area of contracting? 

Panel members responding to this iteration of discussion maintained the Cycle 1 
conclusion that a broad base of experience is desirable. However, none of the experts 
believed that mandating multiple areas of experience was necessary for certification. The 
panelists commented that an individual might not have the authority to determine which 
areas they work in, especially early on in their career. One expert stated that the 
contracting officer of the future might be required to have both contracting and program 
management experience, recommending, “if so, then experience in the corollary field 
would be necessary.” Overall, the group considered a contracting certification to be 
primarily concerned with contracting experience. The panel remained in favor of varied 
contracting experience but would not set a firm requirement for specific areas of 
contracting. 

Question 2. What improvements can be made to these requirements? 

Two respondents stated that the current experience requirements were too lenient for 
a professional certification. Both of these experts sought to extend the number of years 
experience to qualify for certification. One panel member recommended adding an 
examination requirement, believing it would require enough time and preparation for the 
individual to gain enough experience. The other respondent stated an experience 
requirement of at least four years, “the amount of time devoted to training our advanced 
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Copper Caps.” One of the remaining panel members would improve the current 
requirements by reevaluating the role of future contracting officers and requiring 
applicants to meet similar experience requirements; the other recommended adding 
diversity as a stated desire but not a firm requirement. Overall, the panel members 
believed some changes were necessary to current experience requirements for 
certification. The experts were given the opportunity to suggest specific changes to 
certification requirements in Cycle 4. 

Military Responses for Cycle 2c. The response rate for this iteration was 50 percent (3 of 
6 panel members responded). As with the civilian panel, the military experts were asked 
two additional questions concerning experience requirements for professional 
certification in contracting. The questionnaire is presented in full text in Appendix L; a 
synopsis of specific responses is provided in Appendix M. 

Question 1. The APDP certification program has the following experience 
requirements: 1 year in contracting (Level I), 2 years in contracting (Level II), 4 years in 
contracting (Level III). The NCMA CPCM requires 2 years experience in: procurement, 
legal, MIS/IT, inventory mgt, project mgt, production, R&D, marketing, QA, finance, or 
business mgt. Do these requirements support the group consensus that depth of 
experience is important? 

One respondent stated the current requirements did encourage depth of experience 
and were acceptable. Another panel member remarked that requiring an amount of time 
in the career field did not ensure broad experience. This same expert commented that it 
is important to recognize that different positions in contracting require different training. 
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but that current certification requirements are the same for all individuals. This panelist 
adopted another member’s prior suggestion of developing OJT records with specific 
experience requirements for the different types of positions. The third respondent was 
also dissatisfied with the current requirements stating that a certain length of time in the 
career field does not ensure an individual is prepared for the duties of a specific position: 
“I'm looking for someone to get the job done and in some cases the person with 2-years is 
the answer.. .1 don't want some artificial certification requirement to handicap my ability 
to place people into jobs that I know they will excel in.” With such a low response rate it 
was impossible to ascertain an overall group opinion on this matter. The subject was re¬ 
approached in both of the subsequent cycles. 

Question 2. What improvements can be made to these requirements? 

All respondents recommended requiring experience in specific areas, offering 
alternative tasks to meet the same objective in the event an individual is stationed at an 
office that did not offer the opportunity for some specific experience. One panel member 
added that the current requirements were too lenient, that four years was not enough time 
to gain the experience that should be required of an individual certified as Level III 
through APDP. The panel was again given the opportunity to suggest specific changes to 
certification requirements in Cycle 4. 

Civilian Responses for Cycle 2d. The response rate for this iteration was 84 percent (5 of 
6 panel members responded). The panel was asked to comment again regarding the value 
of requiring certification applicants to complete a comprehensive examination. The 
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questionnaire is presented in full text in Appendix N; a synopsis of specific responses is 
provided in Appendix P. 

Question. In what way does requiring the completion of an examination strengthen, 
or fail to strengthen, a professional certification for the contracting workforce? 

The panel members tended to prefer certification programs requiring the successful 
completion of an examination before an applicant is certified. Four of the five 
respondents stated that a comprehensive examination validates the technical competence 
of the individual, requiring them to not only attend class, but to process and apply the 
principles and procedures they were taught. An examination also lends credibility to the 
certification; an individual is not assured certification in a matter of time, they must put 
forth a concerted effort to improve their abilities and knowledge to pass the test. Only 
one expert disagreed, “the process already requires a combination of education, training, 
and experience., .if there is any desire to use examinations, then do it appropriately at the 
specific training course level...” 

Military Responses for Cycle 2d. The response rate for this iteration was 67 percent (4 of 
6 panel members responded). The military panel was asked the same question presented 
to the civilian experts. The questionnaire is presented in full text in Appendix O; a 
synopsis of specific responses is provided in Appendix P. 

Question. In what way does requiring the completion of an examination strengthen, 
or fail to strengthen, a professional certification for the contracting workforce? 

Three of the four respondents remained in favor of a comprehensive examination 
requirement for certification; of the three panelists against an examination requirement in 
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Cycle 1, one responded to this cycle and retained that position. Support for the 

examination referred to the “intense preparation” required, guaranteeing the applicant 

actually learn the craft and not just occupy a position in the career field: “I might have 

spent 4 years in contracting and taken 6 courses, but that doesn’t mean I retained the 

information.” Also, those in favor of an exam requirement believed it identified “those 

who are dedicated to the career field.. .it enhances the contracting community’s 

reputation for professionalism,” and “it causes the individual to spend a significant 

amount of time thinking about what it is they are involved in; to look at relationships 

between different functional specialties. In direct contrast, the remaining expert disputed 

that dedication is proven through testing: 

Reading the above comments, what strikes seems to be the feeling of shared 
pain. By making an examination difficult for a level of certification, it shows 
commitment to the profession to pass. I don’t buy that premise. From my 
perspective, my professionalism and growth in my career field is based upon my 
training opportunities and varied experience; not studying some “school 
answers” to pass a test which bears little semblance to what I may be working 
on or exposed too. 

This panel member believed that completing the required training an experience is 
sufficient for professional development. Of course, the panel stated in previous cycles 
that training and experience requirements as currently written are insufficient for a 
superior certification program. 

Cross-case Comparison. As was the case at the conclusion of the first cycle, the 
researcher benefited from maintaining separate groups by having the opportunity to 
approach several issues raised in Cycle 1 without placing an extreme burden on a single 
panel. The second cycle served to clarify the groups’ positions concerning the initial 
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areas of discussion. However, there were again no distinct points at which one panel 
diverged as a group from the other. Where a majority opinion was evident, that opinion 
was often the majority opinion for both groups. Additionally, Cycle 2 suffered from a 
lapse in the response rate on the part of both groups. Respondents varied between sub¬ 
cycles; since the individual participation within each group was subject to variance, it 
became that much more difficult to draw any conclusions regarding the variance, limited 
though it was, between groups. Establishing a consistent pattern of behavior for each 
group was necessary before the researcher could claim a difference, or lack thereof, in the 
patterns. Based on these circumstances, any future assessment of divergence would be 
seemingly unreliable; the groups had not appeared significantly divergent in the first two 
cycles. Therefore, in an effort to receive a sufficient number of responses on each issue 
so as to maintain the depth of analysis, the researcher decided to combine the participants 
to form a single group for the latter cycles. The common issues between the groups were 
discussed further and unique comments, previously reserved within the groups, were 
provided to foster comment from each demographic represented in the combined group. 

Delphi Cycle 3 

Having determined the civilian and military expert panels had not been significantly 
divergent in their responses during the first two cycles, the researcher combined the 
groups for the two final cycles of discussion. Of a total combined panel size of 12 
experts, the response rate for Cycle 3 was 75 percent (5 civilian and 4 military 
respondents). The experts were provided a questionnaire synopsizing the conversation on 
all previously discussed areas of professional certification. Thirteen additional questions 
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were submitted in an effort to verify an overall group opinion on each subject. The 
questionnaire is presented in full text in Appendix Q; a synopsis of specific responses is 
provided in Appendix R. 

Question 1. The first question addressed formal education requirements. The panel was 
provided a copy of a recent amendment to Section 1724 of title 10, United States Code, 
which required civilian employees and military members of similar occupational 
specialties in contracting to possess an undergraduate degree, with a major or 
concentration in business, before they are granted entry into the career field or before 
they move to another position, for those who were already in the career field. The 
researcher recommended the group complete the discussion on the subject of formal 
education; certification requirements will have to be at least as restrictive as those for 
entrance into the career field. 

The experts had agreed to this point that formal education was valuable, the only 
area of debate centered on whether to require certificants to complete a formal degree 
program. The issue seemed to have been settled by the recent legislation. The panel 
members were offered the opportunity to submit any final comments on the subject. 
Three of the active duty respondents voiced stem opposition to the change stating it will 
be difficult to meet personnel requirements, especially among enlisted positions. One 
was particularly upset stating, “the staff officer at AQ that let the language in the new 
Public Law slip through without bringing it to the attention of AQC [should] be shot.” 
Another military panel member supported the new requirement in part, but disagreed 
with imposing the restriction universally: “The one exception I would make would be for 
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military enlisted. But in their case they will be specifically designated to deploy and 
should focus their training to meet that requirement. None of the civilian members of the 
panel offered additional comments. 

Question 2. How many levels (1, 2, 3, or more) should be included in a professional 
certification program for contracting personnel? Please provide support for your answer. 

Overall, the panel preferred a graduated certification approach consisting of three 
levels; however, that opinion was not unanimous. Three panelists believed that only one 
level of certification should be offered. One expert changed position on the subject from 
earlier cycles, while several respondents remaining with their initial position supported 
their opinion with comments from other panelists. Both of these findings validated the 
Delphi process; had the researcher used another interview method and not provided 
controlled feedback from earlier cycles, it is doubtful any panelists would be convinced 
to change their minds as a result of considering the issues from multiple points of view. 

Questions 3 and 4. Should an individual be required to qualify for a warrant before being 
granted certification? At what level (for those supporting multiple-level certifications)? 

If not, should an individual be required to possess a certification before being granted a 
warrant? At what level (for those supporting multiple-level certifications)? 

The group opinion in this subject was clearly definable; eight respondents did not 
agree with requiring applicants for certification to be qualified for a contracting officer 
warrant. Although the requirements for receiving a warrant are similar to those of 
certification at Level II, the warrant may require additional courses or qualifications 
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based on the particular position the contracting officer will occupy. One of these eight 
experts did not express that a warrant should be a strict requirement for certification but 
should play a role in the certification process; the specific role intended was not offered. 
This issue came to light based on input from a single panelist during an earlier cycle. 
While this expert retained the opinion that a warrant should be required for certification, 
the other panelists were not convinced to support that position. 

Six of eight panelists not requiring a warrant for certification did indicate support for 
the opposite. Several of these respondents suggested individuals be certified at APDP 
Level II prior to being considered for a warrant; a few experts would allow a warrant 
below the simplified acquisition threshold for individuals certified at Level I. Two of the 
panel members sought to keep the two issues completely separate and did not require one 
before the other in either case. 

Question 5. Regarding the number of certification levels you selected in Question 2, 
what PCE courses or subject areas should be required at each level? 

The format of this question allowed for too much variety in answer format for the 
researcher to conduct a meaningful analysis; some respondents provided specific course 
numbers, others only spoke of general subject areas. Three experts responded that the 
current requirements for APDP certification are satisfactory. Three other panel members 
recommended coursework not mentioned in previous cycles; two suggested a program 
management course be added to the curriculum, one saw a need for some basic 
economics instruction. One expert stated that the certification should be based on formal 
education, experience, and a comprehensive examination, and did not call for any 
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mandatory PCE courses. The format of Cycle 4 was more conducive to determining an 
overall group position. 

Question 6. Should all personnel be required to attend the same courses, or should a 
variety be offered for each subject area? 

The panel member not requiring PCE courses abstained from commenting on this 
question. Two of the experts answered that all individuals should attend the same 
courses to ensure everyone meets the same requirements for certification. The remaining 
eight panelists adopted one expert’s suggestion from an earlier cycle; that some 
mandatory courses should be established but electives, possibly in other fields of study, 
should be offered to enhance the breadth of training provided. These experts stated that 
many of the mandatory requirements are not applicable to every acquisition position and 
individuals should be afforded the opportunity to receive relevant training in their pursuit 
of certification. 

Question 7. What format of PCE (CBT, In-Residence, VTC,...) should be required at 
each level? 

The panel member not requiring PCE abstained once again; however, this expert did 
mention that PCE should be encouraged even if not required. All of the remaining 
experts held that at least some courses should be attended in-residence. The group 
seemed in agreement that the interaction between students provided significant value to 
the process. Four panel members allowed for alternative mediums for some, but not all 
course requirements; the other four advised in-residence training for all courses, 
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supporting their positions with comments including, “the learning from one another and 
the networking.. .is just as important as the content of the course taught,” and “the 
learning comes from the interaction of the people; the CBT stuff isn’t really working.” 

Questions 8 and 9. Using the number of levels of certification you selected in Question 
2, what specific experience requirements do you propose at each level? How would you 
measure this experience to ensure the appropriate level of quality and relevance of the 
experience? 

Only two panel members recommended maintaining the current practice of 
measuring experience in terms of years; two other experts would use years of experience 
to state the requirement, but add a qualification that the relevance of the experience 
would have to be documented. Two panel members recommending only one level of 
certification thought experience of at least seven years should be required. Other experts 
offered different expectations at different levels of certification. Overall, the group 
seemed to be interested in higher requirements for experience than those currently 
required; Cycle 4 was a better determinant of that point. The majority of the panel 
believed that specific tasks or duties should be documented before granting certification. 

Question 10. Which, if any, of the above requirements for certification would you be 
willing to waive? Under what circumstances? 

Two experts stated they would not waive any requirements for certification. The 
other panel members offered few waivers and indicated those should only be given on a 
case-by-case basis. The panelists suggested that experience in a related field might be 
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considered to meet the experience requirements, that a particular certification level 
requirement for a certain position might be waived while an otherwise qualified person 
sits in that position, that certain classes might be passed over if an individual’s duties do 
not require that training, and that the degree requirement could be waived for those with 
extensive experience. One panelist proposed that years of experience could be relaxed if 
an individual demonstrated truly remarkable ability. The only class waiver recommended 
was that of exempting the enlisted troops from a degree requirement. 

Question 11. At what level would you require individuals to pass a comprehensive 
examination before granting certification? 

The responses to this question varied widely. Four respondents would not require an 
examination for APDP certification but did not recommend NCMA revise its policy on 
an examination. Of those suggesting only one level of certification, one recommended an 
examination requirement while the other did not specify a preference. One expert 
recommended an examination be added to the requirements for APDP Level III 
certification. One panel member would require individuals to complete an appropriate 
examination at every level of APDP certification. The remaining expert did not indicate 
a specific level at which to require an examination, but did allude to a preference for an 
exam requirement at some point in the process. 

Question 12. Should an individual be allowed to retain their certification status if they do 
not meet the requirements of the Continuous Learning policy? 
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Seven of the nine respondents insisted that individuals maintain their currency in 
training following certification and stated they would revoke certification until the 
requirements of the Continuous Learning policy were met. Two of these panel members 
stated that the acquisition career field changed too frequently to consider an individual 
qualified for certification if they did not maintain current training. The two opponents to 
this opinion were reluctant to penalize individuals who could not attend training as a 
result of funding restrictions at their organizations. One respondent also indicated that as 
an individual achieves higher rank and responsibility, the need for technical expertise is 
replaced by a requirement for managerial and leadership skills not taught through PCE 
courses. 

Question 13. What other requirements should be met in order for an individual to 
maintain their certification status? 

Only one expert recommended additional requirements for individuals to retain their 
certification — periodic revaluation of experience and the relevancy of that experience. 
The remaining eight panel members agreed that no other requirements should be 
necessary. One of these experts did not agree with revoking certification for failure to 
meet continuous learning requirements; effectively, this panel member believed that once 
an individual is certified he or she should retain that certification indefinitely. 

Delphi Cycle 4 

The final cycle of discussion was devised to give the experts the opportunity to make 
final determinations regarding specific certification requirements applying to the APDP 
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and NCMA CPCM certification programs. The panelists were provided the current 
requirements of both programs and asked to recommend changes based on the discussion 
conducted throughout the earlier Delphi cycles. The response rate for the combined 
active duty and civilian panel was 84 percent (10 of 12 panel members responded). The 
questionnaire is provided in full text in Appendix S; tables summarizing the 
recommended changes are provided in Appendix T. 

There were some changes recommended to each element of the APDP certification 
process. One respondent changed position on the number of levels to require, agreeing 
with the majority of the group on a three-level graduated approach to certification. There 
were noticeably fewer changes suggested for the CPCM certification. Overall, the group 
believed the most significant need for change was within the upper levels of the APDP 
process. The panelists recommended more stringent experience requirements, more 
difficult course curriculums, a higher level of education, and some form of re¬ 
certification requirement. A number of panelists stated no change was necessary to 
several elements of the certification; however, discussion in the earlier cycles revealed 
that the group favored changes within training course curriculums and the manner in 
which experience is measured, even if the required courses or length of experience 
remain unchanged. 


Summary 

This chapter described the qualitative data created through the Delphi discussion 
conducted throughout this study. Each question asked of the expert panels was presented 


73 



along with a description of the answers provided by the respondents. This chapter also 
discussed the preliminary conclusions of the researcher as the discussion process 
progressed. The next chapter will discuss the overall conclusions drawn from the 
research. 
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V. Conclusions and Recommendations 


Overview 

This chapter concludes the research report by addressing and answering the research 
questions presented in Chapter I, making recommendations based upon those answers, 
and finally, discussing the limitations of this study and suggesting future research. 

Conclusions 

Research Question 1. What are the strategic goals for certifying acquisition 
professionals? 

The relevant literature reviewed in Chapter II names several purposes for 
professional certification. These objectives serve a variety of stakeholders including the 
profession, the certifying association, the individual professional, and the clientele served 
by the profession. In implementing a professional certification program, the value 
provided to each of these concerns can be neither over nor under emphasized. Over the 
past decade, the defense acquisition community has sustained a myriad of regulatory 
changes in the interest of improving the effectiveness and efficiency with which the DoD 
executes its acquisition processes. The strategic goals behind the Defense Acquisition 
Workforce Improvement Act (DAWIA) and the establishment of the Defense Acquisition 
University (DAU) were rooted in the process reform mandated by regulation and 
expected by the American people. 
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While professional certifications initially serve individuals, providing them with the 
knowledge and expertise that will establish their credentials and enable their career 
progression, there are also benefits to the organization and profession at the strategic 
level. As each defense acquisition professional becomes more qualified and able to 
execute their duties more effectively, the defense acquisition profession becomes better 
equipped to meet the standard of excellence expected. The intention of professional 
certification within the DoD is to ensure each individual is adequately trained, educated, 
and experienced to carry out his or her duties, and to ensure each position is filled with 
the individual qualified to sit in that position, thereby facilitating the highest level of 
competency throughout the workforce resulting in a collective efficiency to meet the 
challenges of improvement imposed on the profession. Expressing these goals in a 
hierarchical manner, the overarching strategic objective behind certification is improving 
acquisition processes for the DoD. This primary initiative is, in part, achieved through 
the professionalization of the workforce, the specific intent of professional certification. 

Research Question 2. To what extent do Air Force acquisition experts believe current 
certification requirements meet or fail to meet those goals? 

The expert participants in this study were all a part of the acquisition workforce 
throughout and prior to the past decade. These experts witnessed the implementation of 
the Air Force’s mandatory certification program as well as the increased encouragement 
to seek previously established, privately administered, professional certifications for the 
contracting career field. The experts found value in every element of professional 
certification programs. Although some valued particular qualifications over others, all 
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agreed that formal education, professional continuing education, and experience 
benefited the workforce and the profession in general. While the experts agree on the 
potential value added to the profession through certification, there is some doubt as to 
whether the APDP certification process currently realizes that full potential. 

The experts were asked questions concerning both the APDP and CPCM 
certification programs; however, the discussion continuously focused on the perceived 
inadequacies of the APDP program while fewer concerns were voiced regarding the 
CPCM. The panel believed the APDP program to be a certification eventually granted to 
all defense acquisition personnel over time. Initially, this does not seem problematic, one 
would think it optimal that all individuals in the field would reach a level of qualification 
to denote certification; however, the experts suggested that the APDP certification was so 
easily obtained, its credibility as an indicator of qualification was becoming diluted. 

Many of the experts believed a professional certification should be reserved for those 
demonstrating the highest level of professional development and dedication, and that the 
APDP certification did not require sufficient effort to assure certificants were any more 
qualified than non-certificants. Based on the opinion of the expert participants in this 
study, it is apparent that the framework of the APDP certification program is appropriate 
to meet the strategic goals of improving the acquisition workforce and defense 
acquisition processes overall, but that the current requirements within that framework fall 
short of a meaningful contribution to the pursuit of those same goals. 
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Recommendations 


Certified Professional Contracts Manager (CPCM) Program. The experts involved in 
this study overwhelmingly supported the National Contract Management Association’s 
CPCM certification requirements as written. The only area of the certification that 
received even moderate scrutiny was the requirement for two years of procurement 
related experience. Half of the experts on the panel suggested the length of experience 
should be at least five years. Two of the more controversial subjects, the examination 
and re-certification requirements, garnered strong support from the panel as currently 
written. Therefore, based on the outcome of this study, the only recommendation to 
NCMA is that it reevaluate the required experience level and consider a more stringent 
stipulation for this element of the certification application. 

Acquisition Professional Development Program. Based on the suggestions of the expert 
discussion panels, it is recommended that changes be considered in the following areas of 
the current requirements: 

Formal Education. The experts placed considerable value on formal education. 
Many of the panel members reiterated that experience and professional training couldn’t 
produce the same benefits as accredited degree programs. Training and experience 
develop an individual’s ability to perform specific tasks, but do not establish the 
foundation for critical thinking essential to learning at both the macro and micro level. 
Over half the experts participating in this study recommended an undergraduate degree as 
a requirement, not subject to waiver, at levels II and III. Several panel members would 
require a Master’s degree prior to granting Level III certification. Should the recent 
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legislative changes to 10 USC, Section 1724 continue in full force and effect, all 
individuals in the career field will possess at least a Bachelor’s degree. 

While, it is recommended the APDP certification requirements follow suit in 
requiring a high level of formal education from those individuals considered worthy of 
receiving a certificate intended to denote professionalism, there is an apparent conflict 
between the desire for professionalism and Air Force mission requirements. If an 
undergraduate degree is required to ensure professionalism, and that professionalism is to 
be mandated for continuing in the career field, many enlisted members will no longer be 
available to meet deployed contingency contracting mission requirements. The expert 
panelists often offered an exemption to allow enlisted members to remain in the career 
field for the sake of those mission requirements. Are those individuals then permitted to 
possess a “professional” certification? If so, does it follow that the completion of an 
academic degree is not necessary for an individual to be a professional? Using Pavalko’s 
description of professions, it is not apparent that a degree represents professionalism, 
only that a specific set of knowledge and skills are required. If it is determined that 
enlisted can still be certified as professionals without completing a degree, how can one 
argue that other personnel must have a business degree to be considered professional? It 
is recommended that this conflict be resolved by allowing enlisted to remain in the career 
field, they are often acquired through the cross training process and provide a valuable 
understanding of military processes in contingency situations, but only offer certification 
to those who meet all requirements established for a “professional” regardless of his or 
her employment status. This would allow continuity in the Air Force’s contingency 
contracting mission, one with a very successful track record, while at the same time 
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preserving the credibility of the certification. Offering waivers to the requirements serves 
to connote an inconsistent interpretation as to the value of the certification. 

Experience. A majority of the experts surveyed in this study believed the experience 
requirements for certification at levels II and III to be too lenient. The fear is that the 
certification will not represent an individual’s actual technical abilities. Individuals can 
spend up to four months attending training courses for the Level I and II certifications, 
and receive the Level II certification after only two years in the career field. When the 
training courses are spread evenly over the two-year period, it is doubtful that an 
individual will have spent more than two consecutive months at a time obtaining the 
necessary work experience for Level II certification. Most of the experts on the panel 
also agree that individuals certified at the highest level, Level III, should represent the 
most highly qualified professional in the career field. The extent and variety of different 
duties carried out by contracting personnel require significant time to master. The 
experts participating in this study did not believe four years to be sufficient time for an 
individual to reach the status that should be carried by the Level III designation. It is 
recommended the APDP certification requirements be revised to require actual work 
experience (apart from training and other duties) for a time exceeding two years for Level 
II, and a time exceeding four years, preferably six years or greater, for Level III 
certification. 

Training. Overall, the experts participating in this study agreed with the current 
course listings required at each level of certification; however, the actual content of 
instruction and administration of those courses was heavily criticized. The PCE courses 
currently offered at DAU are, in the opinion of the group, far less demanding than the 
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need for training dictates. If the APDP is to improve the workforce and therefore 
increase competency across the profession, then the training offered through the program 
should serve a higher purpose than simply completing certification requirements. The 
expert panel held that current courses are next to impossible to fail and rarely challenge 
the student’s current ability upon arrival to the training. The experts also attacked the 
propensity to encourage Computer-Based Training (CBT) as a less expensive alternative 
to in-residence coursework. The experts fear that students will only be required to learn 
objective material and cease to benefit from alternative points of view and the combined 
experiences of other students. Based on the suggestions from this study’s panel of 
experts, it is recommended that DAU reevaluate the strategic purpose for professional 
training and develop measurable indicators to ensure that purpose is fulfilled in the 
workforce. 

Comprehensive Examination. A majority of the experts surveyed did not seek a 
comprehensive examination requirement be added to the APDP process; however, some 
support for the requirement is noteworthy. Those seeking to implement a requirement for 
applicants to complete a comprehensive examination stated the primary reason for doing 
so was rooted in the inadequacies of the PCE currently offered. Several experts agree 
that there is no current requirement for individuals to demonstrate actual learning at 
training courses, they must only attend the courses and retain the information up until an 
interim examination is administered. The experts desire the APDP certification to 
represent a level of professional development, rather than just a record of attendance. It 
is recommended the APDP certification requirements be re-evaluated in this area in order 
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to devise a method, through an examination or otherwise, to ensure the workforce 
actually retains and applies the knowledge and skills representative of professionals. 

Re-certification. Over half of the experts indicated that there was no need to 
periodically re-certify individuals at their current level. However, all of the experts 
believe that continuous learning and development are necessary. Currently, the Air 
Force’s Continuous Learning policy requires individuals to meet annual professional 
education requirements similar to the re-certification requirements of other certification 
programs; however, there is no enforcement of the policy on record. A professional 
certification should represent an individual’s current level of professional development. 

It is recommended the Continuous Learning policy be enforced by requiring individuals 
to meet the policy’s objectives in order to retain their professional status. One method of 
doing so would be to require continuous learning for an individual’s certification to 
remain in effect. 

Summary. DAWIA and the APDP were implemented to ensure the professional 
development and overall improvement of the acquisition workforce. Beyond the benefits 
reaped by the overall defense acquisition process, the professional development of 
acquisition personnel assures both internal customers and the general public that the trust 
they’ve placed in these professionals is well founded. The experts have considerable 
doubt as to whether the current certification requirements actually fulfill these purposes 
of DAWIA. The NCMA CPCM certificate is currently recognized among the experts 
participating here as more rigorous and prestigious than the APDP certification. If that 
opinion is one widely held across the community, it is apparent the level of professional 
development currently required is less than that which is desired. 
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Limitations 


This study examined the state of two certification programs using the Delphi method 
with two panels of experts in the contracting career field. This research was based on 
relevant literature on the subject and conducted in accordance with the principles of the 
method; however, the study does suffer from some limitations. 

The method used to create the data for analysis in this study required open 
discussions between experts who remained anonymous in name and title throughout the 
study. This requirement led the researcher to limit the size of the discussion panels to 
facilitate an efficient exchange of information. The results of this study have limited 
external validity and any conclusions can only be drawn in the context of the expert 
panels surveyed. The results of the research cannot be stated to represent the opinion of 
all experts in the acquisition workforce. 

This study was accomplished under fixed time constraints; therefore, the researcher 
was not at liberty to pursue each issue discovered during the discussion. The study 
offered a broad introduction to the subject matter and identified many issues of high 
importance; however, few of these issues were discussed to the point of group consensus 
and several were passed over as a result of the short time frame. This study does not 
offer a final determination on the subject of professional certification, only a starting 
point for identifying issues in need of further study. 

Finally, it was apparent at different points throughout the discussion, that participants 
read the questions with different predispositions in mind. This resulted in separate 
referent points for the researcher and some participants concerning the terminology used 
in developing questionnaires. At more than one point in the study, the researcher 
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received comments from participants that did not offer the desired information. The 
initial questionnaire in Cycle 1 was pre-surveyed to ensure the inter-item reliability of the 
questions. Beyond the first cycle, questionnaires were based on the comments generated 
during the previous cycle. This process did not lend itself to continual pre-survey 
administrations; it is important to perpetuate the discussion through timely feedback. 
Future researchers should carefully consider the terminology used in developing 
questionnaires for subsequent cycles; clarity should not be overcome by eloquence. 

Recommendations for Future Research 

The questions submitted to the experts in this study often generated more questions 
than answers. The Delphi process, though suitable for the research problem, does require 
extensive time to generate a definable group opinion. This study examined the 
certification process as a whole; however, as the discussion progressed it was evident that 
each element of the discussion process offered an opportunity for a dedicated discussion. 
Additionally, the expert participants all had significantly greater experience in the career 
field than the researcher, these individuals were actually aware of issues the researcher 
had not previously considered. The limitations of this study, and the suggestions of the 
expert panel members, led to the following recommendations for future research: 

The issue of professional certification and workforce improvement remains timely 
and pertinent. It is recommended that future research on the subject include determining 
the effects of the new personnel qualification standards on the acquisition workforce. 

The National Defense Authorization Act of 2001 was mentioned at different phases of 
this study and is an area of great concern throughout the contracting career field. 
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SAF/AQC and the Air Force Recruiting Service would likely benefit from studies 
determining the ability of the current workforce to meet the new requirements and any 
effects to personnel levels and the ability to carry out mission objectives. 

Another issue arising from the discussion was the effectiveness of alternative 
training methods. While the experts preferred the opportunities for interaction offered by 
in-residence coursework, computer based instruction and on-line courses offer less 
expensive means for widespread training. Future research is recommended for 
determining the advantages and disadvantages of the different methods in delivering 
effective acquisition training. The Defense Acquisition University would likely be 
interested in the findings of such research. 

This study focused on the opinions of a few, albeit qualified, individuals concerning 
issues that affect the entire acquisition workforce. In assessing the effectiveness of the 
certification programs, it is necessary to understand the certification requirements were 
designed for implementation in all branches of the armed services. This study looked 
only at the effectiveness of two certification programs as they apply to the Air Force. It 
is recommended that future studies be conducted to determine the effectiveness of the 
programs as they apply to the other branches of the DoD, and to determine whether a 
single set of certification requirements is appropriate for an acquisition workforce divided 
among the different mission objectives of the separate services. 

A final recommendation for future research bom out of this study concerns the 
validity of the current certification programs. Currently, there are no requirements for 
individuals to demonstrate continued mastery of the material presented in PCE courses. 

It is recommended that further study be conducted to determine whether certification 
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programs foster continuous development of the workforce or whether the certificate can 
only indicate an individual’s ability on the date of certification. Also, the extent to which 
routine duties in the workplace continually reinforce initial training or whether periodic 
refresher courses are necessary to ensure continuous workforce improvement. 

Summary 

This chapter discussed the conclusions drawn from the research described in the four 
previous chapters, described recommendations based on those conclusions, and offered 
suggestions for future research on the subject. 

The role of professional certification is widely accepted as a positive motivator of 
professional development. Professional certification enhances the credibility of a 
profession and offers individuals a planned curriculum to improve their technical 
competence and overall professionalism. The APDP and CPCM certification programs 
collectively involve all of the popular elements of successful professional certification 
programs. The CPCM is recognized as a superior certification to APDP as a result of its 
higher level of difficulty and established credibility among acquisition professionals. The 
APDP certification was designed to regulate a mandated improvement to the defense 
acquisition workforce; however, the certification suffers from mediocre requirements and 
a lack of difficulty, making it a poor indicator of an individual’s status as a professional. 
This research is hardly the final word on the subject and was quite limited in scope and 
generalizability. Future studies are necessary to ensure the full benefit potential offered 
by professional certification is realized within the defense acquisition workforce. 
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Appendix A: Expert Panel Membership 


Active Duty Air Force 

1. Colonel Garry Varney 
OC-ALC/PK; Director, Contracting 

2. Lt Col Henry Gaudreau 

DCM Pratt & Whitney; Commander 

3. Lt Col Jeffrey Schmidt 
ASC/YDK; Chief, Contracting Division 

4. Anonymous, Lt Col 

5. Maj or Michael Dodds 

HQ AMC/XPMS; Chief, Strategic Sourcing 

6. Major (Select) Dwayne Sellers 
17 CONS/CC; Commander 


Civilian 

1. Mr. Milton Ross, SES 

HQ AFMC/PK Deputy Director of Contracting 

2. Mr. Joseph Farrey, GS-15 

ASC/PKX Chief, Contracts and Support Division 

3. Ms. Charlotte Dayton, GS-14 

OO-ALC/WMK Chief, Contracts Division, Munitions 

4. Ms. Linda Greaves, GS-13 

1 CONS/LGCD Director, Business Operations 

5. Mr. Charles Woodside, GS-13 

HQ AETC/LGCM Senior Procurement Analyst, 

NCMA National Vice President for Education and Certification 

6. Anonymous, GS-13 
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Appendix B: Initial Information for Participants 


DEPARTMENT OF THE AIR FORCE 

AIR UNIVERSITY (AETC) 


<Date> 

1st Lt Ron Tougaw 
AFIT/ENV 

2950 P Street, Bldg. 640, Box 4107 
Wright-Patterson AFB OH 45433-7765 

<Recipient Address> 



Dear Participant Name> 

Thank you, once again, for offering your time and expertise to participate in this study. This 
research effort, sponsored by the National Contract Management Association (NCMA), will 
utilize the information you provide to benefit Contracting leadership in making future decisions 
concerning our professional certification programs. You have been carefully selected to 
contribute to this project based on your expert-level certification status, as well as the depth of 
your experience in the career field. Your willingness to engage in this forum, and your 
commitment to the continuous improvement of the acquisition workforce is greatly appreciated. 

The purpose and procedures of the study are detailed in Attachment 1. Please complete the 
Participant Profile Questionnaire at Attachment 2 and return it to this office using the envelope 
provided. 

During the week of 13-17 November, you will be contacted via electronic mail announcing 
the start of the study. Additional instructions will be provided to you at that time. If you have 
any questions, or need additional information at any time, please contact me by telephone at 
(937)320-0999 (home), or through electronic mail at Ronald.Tougaw@afit.af.mil. 

Sincerely 


RONALD L. TOUGAW, JR., 1st Lt, USAF 
Master’s Degree Student, 

Air Force Institute of Technology 

Attachments: 

1. Initial Information 

2. Participant Profile Questionnaire 
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Attachment 1 


A DELPHI EXPERT ASSESSMENT OF PROFESSIONAL CERTIFICATION 
PROGRAMS FOR DoD CONTRACTING PERSONNEL 


Purpose: To provide a forum for gathering expert input to benefit Contracting leadership in 
making future decisions concerning professional certification programs available to DoD 
Contracting personnel. 

Background: Approximately 10 years have passed since the Defense Acquisition Workforce 
Improvement Act (DAWIA) required DoD personnel in various acquisition career fields to be 
certified at a level commensurate with the complexities and responsibilities required of their 
position. Since that time, Contracting personnel have been encouraged to enhance their 
professional development by obtaining mandatory DAWIA certifications, as well as other 
optional certifications offered by the National Association of Purchasing Managers (NAPM), 
National Institute of Governmental Purchasing (NIGP), and National Contract Management 
Association (NCMA). In general, obtaining a professional certification involves achieving a level 
of formal education, professional continuing education, and experience in the career field. 
Additionally, some certification programs include the successful completion of a comprehensive 
examination, and require re-certification after an allotted time. 

Method of Study: This study will use the Delphi discussion technique, developed by RAND, to 
conduct an expert evaluation of existing contracting professional certification methods and 
determine the extent to which those methods meet, or fail to meet, the strategic initiatives behind 
the certification process, and assess the reasons attributable to that success or failure. The 
technique allows a panel to convene from geographically separated locations and participate in 
detailed discussions while maintaining anonymity. The panel members’ identities are not 
disclosed to the other members of the panel. This technique provides a forum for discussion that 
is unrestricted by the effects of personality and politics. Panel members are at liberty to speak 
freely and debate openly without attribution or the fear of reprisal. 

Each panel member will receive a set of instructions and an open-ended questionnaire, via 
electronic mail, at the beginning of each discussion cycle. The instructions will set a deadline, 
usually three duty days, for the submission of responses. The facilitator will then compile the 
individual responses and release the results to the individual panel members for comment to begin 
the next cycle of discussion. It is anticipated the study will require five or six discussion cycles. 
Panel members will receive information at a single electronic mail address, but may respond 
using any electronic mail account. NOTE : At no time will an individual response, or summary of 
responses, be attributed to a particular individual participating in the study. 

Point of Contact: Please refer all questions regarding this study to: 

1st LtRon Tougaw 

Ronald.Tougaw@afit.af.mil 
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Attachment 2 


PARTICIPANT PROFILE QUESTIONNAIRE 

Name:__ Grade/Rank:_ 

Office Symbol:_ Duty Phone ( ) 

Address:_ 


Email address you wish to be contacted at:_ 

Current Position Title:_ 

Total Contracting Experience {years): _ 

Operational Contracting {years): _ 

Systems Contracting {years): _ 

Other Contracting {years): _ 

Date of Level III Certification:_ 

Other Certifications {please include date): 

NCMACACM: _ NCMACPCM: 

NIGP: _ NAPM: 

Other: _ 


(.Initials) I hereby give my permission to be identified as a participant in the written report 
following this study. I understand that, at no time during or following this study, 
will my individual responses be attributed to my identity. 

-or- 

(.Initials ) Please do not identify me as a participant in the written report following this study. 


Signature 


Date 
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Appendix C. Delphi - Cycle 1 - Initial Questionnaire 


Introductory Information : 

Approximately 10 years ago the Defense Acquisition Workforce Improvement Act 
(DAWIA) significantly changed the way the DoD components manage acquisition 
personnel career development. DAWIA established mandatory standards for certifying 
individuals at the appropriate level for their position. Additionally, acquisition personnel, 
those in the contracting career field in particular, have been encouraged to pursue various 
outside certifications offered by NAPM, NIGP, and NCMA. Each certification program 
is composed of various requirements, some in common and some unique. Over all, 
contracting professional certification programs require a level of accomplishment in some 
or all of the following categories of development: Formal Education, Training 
(Continuing Professional Education (CPE)), Contracting Experience, and Successful 
Completion of Comprehensive Examination. The Air Force’s APDP certification 
program separates individuals into one of three levels of certification, while the other 
programs offer single certifications or multiple certifications independent of one another. 
Some of the certification programs require individuals to be re-certified through 
additional CPE or examinations, while others have no such requirement. 

The following questions concern the certification of contracting professionals in general. 
The term “certification ” does not indicate a particular certification program; it only 
refers to the act of determining an individual has achieved a desired level ofprofessional 
development in the career field. 

Questions : 

1. What value does formal education provide to the workforce that cannot be achieved 
through experience and training? What specific degrees or disciplines should be 
required? At what level (BS, MS, PhD ...)? 

2. The training requirement of most certification programs is accomplished through the 
completion of PCE courses. What value, if any, does this type of training offer beyond 
that received at the workplace? What specific courses or subject areas should be 
required? 

3. What is the value of requiring a minimum amount of c ontracting experience before 
granting certification? What is the appropriate way to measure that experience? What 
specific tasks or areas of experience should be required? 


4. What is the value added by requiring the successful completion of a comprehensive 
examination before granting certification? 

5. What value is provided to the workforce by requiring re-certification ? Which 
categories of development should be included in the re-certification requirement? 
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6. What is the value of certifying professionals at progressive levels , as opposed to 
providing a single certification for all personnel? 

7. Please review your answers to the questions above. Should there be different 
requirements for military and government civilian personnel? For DoD and private 
industry personnel? 

End Cycle 1 . 
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Appendix D. Synopsis of Responses For Delphi Cycle 1 


Table 8. Civilian Responses to Delphi Cycle 1 / Question 1 

What value does formal education provide to the workforce that cannot be achieved 
through experience and training? 6offpanelists^ responding^ _ 

Formal education supplies a broad base of learning, well-rounded personnel; draws on 
topics not focused on in training or achieved through experience. (3) 

Formal education adds credibility and credentials.. .society values academic degrees and 
equates them with competence and success.. .for society to consider federal employees to 
be the most qualified in their field, academic degrees are necessary. (2) 

Formal education supports a wider worldview.. .guarantees a better understanding of the 
business world.. .facilitates a network that can be utilized in the work world. 

Experience is very important to day-to-day government business, but the perception of 
the quality of government personnel is just as important. 

What specific degrees or disciplines should be required? 4o£6_ panelists responding 

Earlier in my career, I would have answered that a business degree had a significant 
advantage.. .1 have seen employees without formal business education pick up on 
business principles through the application of common sense and government training 
and become very competent in contracting. 

In the systems and logistics arena, an undergraduate degree in Industrial Management is 
quite helpful.. .for operational contracting, a general business degree is sufficient.. .all 
programs should include training in cost accounting. 

A bachelor degree, or equivalent, in any discipline should be acceptable. 

Although business should be part of the requirement, what other disciplines could bring 
to the career should not be overlooked.. .history and other degrees teach people to 
research and think.. .a well-rounded education is very important. 

At what level (BS, MS, PhD...)? 4 o£6_ panelists^ responding _ 

Grades 11-12 should have a BS/BA; above Grade 13, an MS should be required. 

An individual whose undergraduate degree is not in business would benefit from an 
MBA program or specialized MA or MS procurement program. 

Beyond a bachelor’s degree, additional education should not be required, but should be 
considered favorably when hiring for jobs with increased responsibility. 

As a minimum, a BS degree should be required. Individuals without a Master’s Degree 
will not be competitive for promotions to Grade 13 or above. 

Numbers in parentheses indicate the number of similar responses 
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Table 9. Military Responses to Delphi Cycle 1 / Question 1 


What value does formal education provide to the workforce that cannot be achieved 
through experience and training? 6 of 6 panelists^ responding • __ 

Formal education exposes individuals to a wider variety of topics and learning situations 
than normally possible though experience in a work environment. (3) 

Contracting cannot consider itself a professional community without a degree 
requirement. 

Individuals completing degrees prove their ability to see a long-term task through to 
completion. 

Formal education aids in developing the type of new and innovative thinking required for 
success in the career field; it develops an individual’s ability to critically analyze and 
apply information that is produced by others. 

Formal education provides opportunity for people to learn how to learn, improving their 
ability to apply logic to develop solutions to problems and helping them to develop an 
appreciation for different perspectives. 

What specific degrees or disciplines should be required? At what level (BS, MS, 
PhD...)? 6of6 panelists responding • _ 

The type of degree should be business or business related (or include a minimum number 
of business credit hours).. .a business degree covers many important areas such as 
business law, business communication, organizational structure, management and 
leadership skills.. .how can we expect our career field to understand contractors without a 
degree in business?.. .business related degrees are the most desirable at the undergraduate 
level.. .but would not rule out other degrees as long as the person has at least 24 credit 
hours in business. (4) 

[In systems/logistics contracting] it seems the FAR is used more as a starting place for an 
eventual deal. The contracting team is challenged with creating a contract that captures 
the deal yet complies with law.. .new thinking is required. The thinking pattern 
encouraged by a liberal arts degree aids in that type of thinking.. .1 don’t believe it has to 
be a business degree, in fact, I almost think that a bachelor’s in business is a bad thing. I 
am convinced an engineering degree provides a person too dogmatic., .no objection to 
business degrees at the Master’s level. 

Almost any undergrad degree could achieve the desired objectives. At some level, the 
duties of contracting personnel include the supervision of others. A graduate degree that 
emphasizes organizational and interpersonal relations (e.g., management, psychology, 
political science, sociology) may be beneficial. 

Numbers in parentheses indicate the number of similar responses. Some respondents 
_ may be represented^ by multiply responses _ 
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Table 9. (Continued) 

What specific degrees or disciplines should be required? At what level (BS, MS, 
PhD...)? Continued^ __ 

To enter the career field it should be a BS. To advance to senior level we should require 
an MS. 

A bachelor’s degree should be required. Additionally, a specific amount of business 
courses should be a part of the degree. You should not be required to have the business 
hours to enter into the career field; however, in order to move up to jobs of more 
responsibility, the business courses should be required. 

In order to attain top certification, an undergraduate degree should be required. 

One respondent stated an exemption of formal education requirements for enlisted 
personnel, “it's well proven that enlisted personnel perform post, camp, and station 
contracting duties in an outstanding manner.. .they are not required to have a college 
degree. ..Iam always pleased when I hear an Army officer remark that the Army sends 
company officers and even Majors to do the same work as our enlisted. 
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Table 10. Civilian Responses to Delphi Cycle 1 / Question 2 


What value does PCE training offer beyond that received at the workplace? 

6of6 panelists, responding^ _ 

PCE develops needed skills.. .training in the mechanics of the trade. (2) 

PCE exposes an individual to contracting methods and ideas they might not be ordinarily 
exposed to.. .allowing for a broader academic theoretical discussion. (2) 

PCE reinforces good on-the-job training. 

PCE offers the opportunity to network with peers and draw on each other’s 
experiences. (2) 

PCE catches current trends.. .is the best way to get new ideas out to the field. (2) 

What specific courses or subject areas should be required? 

2of6 panelists, responding. _ 

The following courses or topics should be required: Contract Law, Pricing, Contract 
Administration, Negotiation Techniques, and Market Research. 

A requirement for a Quantitative Methods course should be added. 

Broader training requirements should be added, “we need to have training focused on 
other disciplines (finance, history, etc.) to help the workforce see the big picture.” 

Other Issues. 3 of 6 panelists^ contributing^ _ 

Some questioned whether certification programs have heightened the need for PCE, 
“prior to the APDP certifications in contracting, virtually 100% of our workforce had 
completed more than the requisite courses.. .certification requirements have had little to 
no impact on the PCE training or workforce receives,” and “[once an individual is 
certified] the availability of government sponsored courses is very limited...it forces 
people to seek out organizations, such as NCMA, who can provide these courses.” (2) 

Another stated that, “if [PCE] is received before on-the-job training, the individual may 
have difficulty in finding the correct frame of reference for the topics being taught.” 

Numbers in parentheses indicate the number of similar responses 
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Table 11. Military Responses to Delphi Cycle 1 / Question 2 

What value does PCE training offer beyond that received at the workplace? 

6of6 panelists responding^ _ 

PCE courses allow students to focus completely on a given subject without the 
interference of other duties. (2) 

PCE exposes an individual to contracting methods and ideas they might not get exposed 
to during routine OJT.. .offers a wide range of scenarios and explores different options to 
resolve issues. (3) 

PCE allows people to acquire specific knowledge and skills...it is important for those 
entering the field to learn the tools necessary for the profession. 

Offers the opportunity to network with peers and draw on each other’s experiences. (6) 

What specific courses or subject areas should be required? 

3 of 6 panelists^ responding _ 

The basic courses as currently structured offer the desired curriculum; it’s a good mixture 
of general contracting to specific contracting issues. (2) 

The following course work should be required: Source Selection/Best Value, Contract 
Management, Contract Law, Services, Construction, Systems, and Contingency 
Contracting, Pricing, and Electronic Contracting. 

Other Issues. 1 of 6 panelists^ contributing^ _ 

I’m not sure the majority of training is still accomplished by sending people to PCE. I 
believe we should have the courses we had ten years ago.. .pricing was done in three 
phases, each [one] difficult, but people learned to price.. .we had specific courses on 
services contracts.. .we used to send everyone to contract law.. .we had three courses in 
how a major program was conceived and brought through the milestones. Now we just 
tell people to sit in their cubical and do [computer-based training] and “oh, by the way, 
answer your phone while you’re there”. 

Numbers in parentheses indicate the number of similar responses 
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Table 12. Civilian Responses to Delphi Cycle 1 / Question 3 


What is the value of requiring a minimum amount of contracting experience before 
granting certification? 5_of6 panelists responding _ 

Without performing the task, mastery will never be achieved. 

Education and training are only effective when put into practice.. .most contracting 
knowledge comes from the job, not the classroom. (2) 

Experience provides the ability to screen distracting and irrelevant information to 
discover the root of the problem.. .experience also involves dealing with political and 
emotional issues not always present in straightforward classroom exercises. 

Requiring experience before certification allows the assumption of a level of technical 
competency, unlike academic degree programs. 

What is the appropriate way to measure that experience? 3 of ^panelists responding 

The current method of measurement is appropriate. Currently, professional certification 
programs require a number of years experience in contracting or a contracting related 
field. 

“X number of years in a certain job could be a way of measurement.” This response is 
similar to the first but seems to require experience in specific areas of contracting rather 
than just a minimum time in the career field. 

The current measure is appropriate but the level of experience required is insufficient, 
“experience time should equal that of the complexity of courses required for that 
certification level.. .a minimum of 2 years experience for Level 1.. .6 years for Level 2 
and 10 years for Level 3 would be more in line with the course material.” Note: The 
levels referred to are those of the APDP certification program. 

The individual should meet the requirements necessary for obtaining a PCO or ACO 
warrant before being granted certification. 

What specific tasks or areas of experience should be required? _ 

0 of 6 panelists contributed information in response to this portion of the question. 

Numbers in parentheses indicate the number of similar responses 
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Table 13. Military Responses to Delphi Cycle 1 / Question 3 

What is the value of requiring a minimum amount of contracting experience before 
granting certification? 5_of6_ panelists responding^ _ 

Proficiency only comes through solving real problems. 

Certification denotes a level of professionalism and competency.. .standards for 
experience must be set to ensure the integrity of the certification process. 

The knowledge and skills obtained through classroom training can only be matured 
through application to actual contracting processes...no amount of education is sufficient 
without hands-on contracting experience. (4) 

What is the appropriate way to measure that experience? 6 o[6_panelists, responding 

The contracting position can be broken down into a number of essential tasks. Tasks can 
then vary in terms of their relative effect on organizations (e.g., criticality, dollar value, 
priority). Experience should be measured in terms of the quantity of essential tasks 
performed of varying quality 

Experience should be measured by time spent in a specific job.. .years spent in a SPO vs. 
operational contracting vs. administration. To rise to the senior levels in contracting a 
person should have a background in [all three areas]. 

The enlisted OJT skills upgrade is the best way I’ve seen.. .a list of core tasks that any 
contracting professional would need to know. This system is susceptible to “pencil 
whipping”; however, you have to rely on the integrity in the system and trust the 
individuals certifying an individual. 

A base of knowledge must be obtained for each level of certification. [Basic level 
certificants] must understand and have accomplished 1 year of simplified acquisition 
procedures. [Intermediate level certificants] must demonstrate understanding and 3-4 
years experience in services, construction, or systems contracting. [Advanced level 
certification] should require 8 years of above experience. 

The only way to ensure an individual has mastered the necessary skills to perform all job 
requirements is to give that individual work in every area of the business.. .this is not 
practical. The default is to grant certification after a minimum amount of time on the 
job.. .this is by no means an optimal solution, either. 

I don’t think it appropriate to segregate by type of contracting. How do you classify the 
experience? Some contracting in ASC is more like base contracting and some at many 
HQs is more like systems. Just go with the number of years. 

_ Numbers in parentheses indicate the_ number of similar^ responses^ _ 
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Table 13. (Continued) 

What specific tasks or areas of experience should be required? _ 

I believe that the career field should require some minimum job experience in addition to 
time before certification. For example, a person should have experience in solicitation 
preparation, contract negotiation, contract award, etc. 

I believe a standardized base of knowledge through experience must be obtained for each 
level of certification. For example, Level I must understand and have accomplished one 
year of simplified acquisition procedures. Level II must demonstrate understanding and 
three to four years experience in services, construction, or systems contracting. Level II 
would require eight years of the above experience. 

Numbers in parentheses indicate the number of similar responses 
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Table 14. Civilian Responses to Delphi Cycle 1 / Question 4 

What is the value added by requiring the successful completion of a comprehensive 
examination before granting certification? 6of6 panelists^ responding^ _ 

It would not be a “professional” certification without a comprehensive examination. For 
Level I certification, I don’t believe a test is necessary - training and experience complete 
this. Level II is about the same. However, having a test for Level II and/or III may help 
in declaring contracting a professional field. 

A comprehensive examination gives credibility to the certification. If there is no exam, 
there is not confirmation that the individual has mastery of the material.. .only that the 
individual was in the classes or was at work for a specified period of time. For the 
certification to have credibility there must be a comprehensive exam and it must be 
possible to fail the exam. The professional certifications granted by AFIT in the past 
aren’t real prestigious. Everybody (well almost everybody) passes the classes and 
attaining the certification isn’t much of an accomplishment. The CPCM exam 
administered by NCMA, on the other hand, had a 60% fail rate in its early days. 
Successfully passing the CPCM is truly an accomplishment and is recognized as such by 
the professional workforce. 

The requirement forces the person to study areas that they would not normally deal with 
on an everyday basis. The contracting professional should at least be exposed to all 
aspects of the field and the comprehensive exam does this. With contracting becoming a 
dynamic field one never knows where or what one will be doing next year. 

If formal education, training, and experience are required, then examinations shouldn't 
be. Only consider examinations to substitute or waive formal education. 

There is little alternative if the goal is for the candidate to show mastery of the subject 
material 

Comprehensive examinations are overvalued. We have study programs that tend to teach 
the test. Not optimal. 
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Table 15. Military Responses to Delphi Cycle 1 / Question 4 

What is the value added by requiring the successful completion of a comprehensive 
examination before granting certification? 6oj_6_panelists responding _ 

I am not in favor of a comprehensive examination for granting certification at this stage 
in the ADPD program. PCE test scores, OJT evaluation, appraisal feedback, etc., are 
adequate standards for determining whether or not an individual is able to perform the 
job. I would be willing to reconsider this position, if and when OPM changes the status 
of contracting to a professional series. 

Comprehensive exams serve two purposes. The first and most obvious is reason is to 
assure a minimum standard for entry into the profession. The other second reason 
concerns increasing the level of commitment of the individual to the profession. One of 
the primary predictors of commitment to a course of action is the amount of effort 
required to initiate that course of action. People who have struggled to complete a 
comprehensive examination will have greater commitment to the values and beliefs held 
by that professional field. 

It demonstrates a minimum level of knowledge. It also shows dedication to the career. 
Just like accounting uses the CPA to distinguish between accountants. 

I’m not certain the examination and study for the examination are discriminators. But, 
the demonstration of commitment shown by studying for and taking the examination is 
what I look for. 

I don’t see the value of any comprehensive exam for government certification. My 
experience of being around folks preparing for NCMA certification is to just study to 
take/pass test. I’m not sure that requirement alone for NCMA certification equates to an 
instant professional. 

A comprehensive examination is important because it indicates the overall knowledge an 
individual has in contracting. It’s one thing to know about a specific area like source 
selections, but every contracting professional should have a good general knowledge of 
all contracting areas. You have to be able to see the forest through the trees. 
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Table 16. Civilian Responses to Delphi Cycle 1 / Question 5 

What value is provided to the workforce by requiring re-certification? 

6_of_6 panelists^ responding _ 

The objective of re-certification is to ensure the workforce remains current concerning 
regulatory changes and best practices in the profession. (2) 

Individuals should be required to continue their professional education and training even 
after they achieve certification. (5) 

Re-certification is of greater benefit to the certifying association (keeps them in business) 
than it is to the workforce. 

I do not see any value, as long as ongoing training is required. 

I see no value in re-certification if a graduated/progressive certification approach is used. 

Which categories of development should be included in the re-certification 
requirement? 3 of 6 panelists responding^ _ 

Generally, they should be trained in areas that have been subject to substantial change. 

Include courses in contract and fiscal law as well as information on other government 
agencies and how they operate. 

At least a working knowledge of any changes in the law, regulations, and industry 
standards that affect the profession. 

Numbers in parentheses indicate the number of similar responses. Some respondents 
_ may be represented by multiple^ responses^ _ 


103 





Table 17. Military Responses to Delphi Cycle 1 / Question 5 

What value is provided to the workforce by requiring re-certification? 

6 off) panelists^ responding _ 

A re-certification requirement will encourage people to take time from their busy work 
schedules in order to attend professional continuing education and may motivate people 
to maintain their tool set. Sometimes this type of incentive is needed. People tend to do 
the things they are good at. This often comes at the expense of other things that may be 
important to the position. 

Absent a comprehensive examination, I am not in favor of re-certification given the 
current standards of education, training and experience. I am also not in favor of 
subjective standards for re-certification—it will become a paper exercise. 

Keeping current through training and meeting a minimum standard (i.e., 40 hours per 
year of acquisition related coursework) is of value for re-certification. This requirement 
ensures the employee continues to be exposed to current trends in acquisition and doesn’t 
remain static in their knowledge base. 

Contracting is changing constantly. What I learned 11 years ago in contracting has 
change 3 or 4 times since then. Re-certification is important to ensure people are keeping 
up with these changes. 

I don’t believe there is any value in re-certification. For example, my certification 
(CPCM) has expired. I had spent over two years as a division chief at ASC and over two 
years at the AFMC Headquarters when it expired. Yet, I needed to send seventy dollars 
and submit a list of NCMA seminars I’d sent. Seemed like method to get me to send 
money and demonstrated no value added. 

Which categories of development should be included in the re-certification 
requirement? 2of6_ panelists_ responding^ _ 

Re-certification can include continuing education, experience, and knowledge/experience 
captured in some form of examination. 

I think all categories of development should be included. All levels of contracting are 
constantly changing. 
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Table 18. Civilian Responses to Delphi Cycle 1 / Question 6 

What is the value of certifying professionals at progressive levels, as opposed to 
providing a single certification for all personnel? 6of6 panelists responding _ 

Each level demonstrates progress in career, professional standing and, hopefully, 
knowledge in the field of study. 

Certifying individuals at progressive levels gives new personnel attainable goals for the 
short-term, rather than overwhelming them with long-term expectations. The APDP 
graduated approach is effective in eventually producing contracting masters. (2) 

Individuals progress in the career field from apprentice to journeyman to master 
craftsman. An individual should be required to show they have completed the 
requirements of each level before being allowed to progress to the next. 

Having a lower level certification (e.g., the CACM rather than the CPCM) is more of a 
mark against than a mark for an individual. It indicates an individual is not yet proficient 
enough for the top certification but received a token certification for lesser requirements. 

Contracting work does not fall out at logically varying degrees of difficulty. The 
certification should match the level of work required. It is not always easy to determine 
which individuals need a higher level of certification. 

Numbers in parentheses indicate the number of similar responses 
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Table 19. Military Responses to Delphi Cycle 1 / Question 6 

What is the value of certifying professionals at progressive levels, as opposed to 
providing a single certification for all personnel? 6of6_ panelists responding^ _ 

The primary value of progressive levels is to provide intermediate goals to the individual 
entering the profession. A certification program with a meaningful set of requirements 
may appear daunting to someone just entering a career. People need to believe that their 
efforts will lead to success. If they have low expectations of this, they may not try. By 
providing progressive levels, the requirements for certification become more in line with 
the individuals expectations. 

I am in favor of “progressive levels” in the contracting career field because of the 
stratification of jobs in the career field. There are many jobs that require minimal 
contracting training and experience. We set the standards for those positions and should 
recognize those individuals who have met those standards. They should not be held to a 
“higher” standard that is not necessary to perform their job. 

It allows me to meet a person with an expectation of their level of knowledge. In 
addition, it allows me to understand an employee’s motivation. Lastly, it gives me a 
check, years in the career field vs. certification level; if the two don’t match I’ve probably 
got a person that I don’t want to hire. 

Progressive levels are desirable because employees have different levels of responsibility 
and are exposed to variable degrees of difficulty. One size fits all is seldom the right 
approach and doesn’t fit the reality of type of workload distributed within an 
organization. A certification should fit the person’s expertise, experience and complexity 
of workload expected to be handled. 

Having progressive levels gives people the ability to establish short-term goals and fulfill 
these goals. If you have a single certification that takes 4 to 6 years to attain, you may 
lose many people before they ever started. 
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Table 20. Civilian Responses to Delphi Cycle 1 / Question 7 


Please review your answers to the questions above. Should there be different 
requirements for military and government civilian personnel? For DoD and private 
industry personnel? 6of6_ panelist^ responding _ 

I see no reason for different requirements for military vs. Government Civilian vs. private 
industry personnel. One rigorous certification level is all that is required if you are to 
maintain the credibility of the certification. Creating different requirement levels will 
result in one certification category being seen as having lesser requirements and therefore 
lesser prestige/credibility. 

Requirements for certification in the federal government for contract professionals should 
be the same for the military and the civilian workforce. That does not mean that the 
military that are required for deployments should not carry a separate certification level 
denoting their skills and competency levels as contingency contracting officers. They 
are a critical part of the performing the DOD mission.. .Since [DoD] contracting deals 
directly with our counter parts in industry, I think the certification designations should 
mirror that process also. Maybe we should tie the Level 3 APDP certification to the 
passing of the NCMA CPCM exam. It would really cut down on the Level 3 population. 

Military and civilians should have the same requirements.. .1 don't believe DoD and 
private industry always have the same acquisition focus, and an exact duplication in 
certification would be difficult. 

It will be very difficult to apply all the criteria to enlisted personnel who carry much of 
the load in being deployed for contingency contracting efforts. Few of them could meet 
all the qualifications. Unless and until it becomes necessary to have a license to be a 
contracting professional (improbable, at best) there is no need to include private industry 
personnel in the discussion. 

Military and civilian members should follow the same rules of the road - no exceptions. 
DoD employees have a public fiduciary responsibility that calls for a higher degree of 
professionalism than industry would be expected to levy on itself. 
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Table 21. Military Responses to Delphi Cycle 1 / Question 7 


Please review your answers to the questions above. Should there be different 
requirements for military and government civilian personnel? For DoD and private 
industry personnel? 6of6_ panelists, responding _ 

Absolutely not, there should not be different requirements for military and government 
civilian personnel. An argument might be made that civilians do not do contingency 
contracting; however, I disagree with that. Contingency contracting takes many forms 
that could occur right at the base you’re at (i.e., natural disaster). Military and civilians 
perform the same tasks and should have the same requirements. 

DoD and private industry personnel should have different requirements. I define private 
industry as companies that do not contract with the federal government. This part of 
private industry has different codes they comply with and should have different 
requirements. Private industry that contracts with the federal government should have 
the same requirements as the DoD as they have to comply with substantially the same 
regulations and law as DoD personnel. 

No difference in certification for any gov. employee. There may be need to be a study if 
different certification requirements should be enacted for DOD and private industry. 

There are different standards and expectations between the 2 groups. 

Military should be required to attend more contracting schools, in residence, which deal 
with contracting in deployed locations. All DoD contracting personnel should attend 
more classes in residence. Private industry is an entirely different animal. It even 
depends upon a different set of laws, the Federal Acquisition Regulations versus the 
Uniform Commercial Code. Private industry depends more upon control and oversight 
of subcontractors than DoD. Private industry uses long-term agreements and other 
concepts that DoD cannot. 

I believe this has been a big problem in the past. The military have been held to a higher 
standard. Military officers have to have a degree, get a master’s, and move to a variety 
of jobs just to get promoted. In order to make contracting a truly professional career 
field, civilians must be required to have a degree. If civilians want to hold senior level 
positions they must be willing to move in order to get a broad background in contracting. 
However, I recognize the most of the civilians will not move, and that’s fine. The 
civilians still must provide continuity. I just think we must now weed-out all civilians 
who do not have a degree. 

I do not believe there should be different standards just because of the uniform a person 
wears. I am not concerned about private industry requirements. 

I think the requirements should be the same. I believe the job duties for a DOD 
contracting professional are independent of military status. 
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Appendix E. Delphi - Cycle 2(a) - Formal Education (Civilian) 


What value does formal education provide to the workforce that cannot be achieved 
through experience and training? What specific degrees or disciplines should be 
required? At what level (BS, MS, PhD ...)? 

I’ve consolidated your responses to the first question. The number in the parentheses 
following each statement indicates the number of you who made similar responses. 

There was general agreement to the value of formal education, but some disagreement on 
the discipline and level of education. In this cycle, we need to make sure we have 
consensus on the areas of agreement, and then further explore areas where there is 
dissimilarity in opinions. 

The first section contains points of relative agreement. I want to make sure we have 
consensus on these points. Please comment on the extent you agree or disagree with each 
statement. Provide your reason(s) for disagreement. 

Areas Of Agreement : All respondents believe that formal education provides value to 
the workforce. The support offered for this assertion included: 

- Formal education supplies a broad base of learning, well-rounded personnel; draws 
on topics not focused on in training or achieved through experience. (3) 

- Formal education adds credibility and credentials.. .society values academic 
degrees and equates them with competence and success.. .for society to consider federal 
employees to be the most qualified in their field, academic degrees are necessary.(2) 

- Individuals with formal education are familiar with methods of analyzing and 
solving problems.. .they are able to apply analytical and organizing techniques on the job. 

- Formal education supports a wider worldview.. .guarantees a better understanding 
of the business world.. .facilitates a network which can be utilized in the work world. 

- Experience is very important to day-to-day government business, but the perception 
of the quality of government personnel is just as important. 

Consensus seems to have also occurred on the issue of the disciplines to require for 
formal education. Overall, the group agreed that degree requirements need not be limited 
to a business curriculum. 

- Earlier in my career, I would have answered that a business degree had a significant 
advantage.. .1 have seen employees without formal business education pick up on 
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business principles through the application of common sense and government training 
and become very competent in contracting. 

- A bachelor degree, or equivalent, in any discipline should be acceptable. 

- In the systems and logistics arena, an undergraduate degree in Industrial 
Management is quite helpful.. .for operational contracting, a general business degree is 
sufficient.. .all programs should include training in cost accounting. 

- Although business should be part of the requirement, what other disciplines could 
bring to the career should not be overlooked.. .history and other degrees teach people to 
research and think.. .a well-rounded education is very important. 

Areas for Further Discussion : There were different thoughts on the level of education 
that should be required. Overall, it is not clear whether the group believes the formal 
education requirement must be met through the completion of a degree or a lesser amount 
of coursework. 

- Grades 11-12 should have a BS/BA; above Grade 13, an MS should be required. 

- An individual whose undergraduate degree is not in business would benefit from an 
MBA program or specialized MA or MS procurement program. 

- Beyond the bachelor degree, additional education should not be required, but 
should be considered favorably when hiring for jobs with increased responsibility. 

- As a minimum, a BS degree should be required. Individuals without a Masters 
Degree will not be competitive for promotions to Grade 13 or above. 

Questions : 

1. Why is it important for the individual to possess an undergraduate degree before 
being granted entrance into the contracting career field? 

2. Considering APDP and other contracting professional certifications (CPCM, 
CACM...), why is it important for the individual to possess an undergraduate 
degree before being granted professional certification? 

3. Under what circumstances would a formal education requirement be waived? 


4. Does your answer to Question 1 apply equally to officer, enlisted, and civilian 
personnel? Why or why not? 


End Cycle 2a . 
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Appendix F. Delphi - Cycle 2(a) - Formal Education (Military) 


What value does formal education provide to the workforce that cannot be achieved 
through experience and training? What specific degrees or disciplines should be 
required? At what level (BS, MS, PhD ...)? 

Eve consolidated your responses to the first question. The number in the parentheses 
following each statement indicates the number of you who made similar responses. 

There was general agreement to the value of formal education, but some disagreement on 
the discipline and level of education. In this cycle, we need to make sure we have 
consensus on the areas of agreement, and then further explore areas where there is 
dissimilarity in opinions. 

The first section contains points of relative agreement. I want to make sure we have 
consensus on these points. Please comment on the extent you agree or disagree with each 
statement. Provide your reason(s) for disagreement. 

Areas Of Agreement : All respondents believe that formal education provides value to 
the workforce. The support offered for this assertion included: 

- Formal education exposes individuals to a wider variety of topics and learning 
situations than normally possible though experience in a work environment. (3) 

- Contracting cannot consider itself a professional community without a degree 
requirement. 

- Individuals completing degrees prove their ability to see a long-term task through 
to completion. 

- Formal education aids in developing the type of new and innovative thinking 
required for success in the career field; it develops an individual’s ability to critically 
analyze and apply information that is produced by others. 

- Formal education provides opportunity for people to leam how to learn, improving 
their ability to apply logic to develop solutions to problems and helping them to develop 
an appreciation for different perspectives. 

Areas for Further Discussion : 

The second half of the question did not result in as much across the board consensus. 
Several different opinions arose concerning the need to require a degree for entering the 
career field or granting certification. 
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- 1 of 6 respondents did not require a degree for entrance into the career field, "you 
should not be required to have the business hours to enter.. .in order to move up to jobs of 
more responsibility, the business courses should be required." 

- 1 of 6 respondents stated that an undergraduate degree should be required for 
entrance “to enter the career field it should be a BS” 

- 1 of 6 respondents stated that an undergraduate degree should be required for 
certification at higher level “in order to attain top certification, an undergrad degree 
should be required” 

- 1 of 6 respondents stated that an exemption of formal education requirement for 
enlisted personnel, “it's well proven that enlisted personnel perform post, camp, and 
station contracting duties in an outstanding manner.. .they are not required to have a 
college degree. ..Iam always pleased when I hear an Army officer remark that the Army 
sends company officers and even Majors to do the same work as our enlisted...” 

- 2 of 6 respondents stated that a graduate degree should be required for certification 
at highest level (2) “to advance to senior level we should require an MS,” and .. .“at some 
level, the duties of contracting personnel include the supervision of others.. .a graduate 
degree that emphasizes organizational and interpersonal relations (e.g., management, 
psychology, political science, sociology) may be beneficial” 

In addition, some respondents propose that any degree can provide the needed value to 
the workforce, while others stated a specific degree preference. 

- 4 of 6 respondents stated that the type of degree should be business or business 
related (or include a minimum number of business credit hours).. .“a business degree 
covers many important areas such as business law, business communication, 
organizational structure, management and leadership skills.. .how can we expect our 
career field to understand contractors without a degree in business?” .. .“business related 
degrees are the most desirable at the undergraduate level.. .but would not rule out other 
degrees as long as the person has at least 24 credit hours in business.” 

- A contrasting opinion expressed the matter this way: “[In systems/logistics 
contracting] it seems the FAR is used more as a starting place for an eventual deal. The 
contracting team is challenged with creating a contract that captures the deal yet complies 
with law.. .new thinking is required. The thinking pattern encouraged by a liberal arts 
degree aids in that type of thinking.. .1 don’t believe it has to be a business degree, in fact, 

I almost think that a bachelor’s in business is a bad thing. I am convinced an engineering 
degree provides a person too dogmatic.. .no objection to business degrees at the Master’s 
level.” 

- The final respondent did not specifically require a business degree, stating that 
almost any undergrad degree could achieve the desired objectives. The respondent did 
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indicate the degree should prepare the individual for contracting by requiring proficiency 
in oral and written communication, and interpersonal and organizational skills. 

Three areas of disagreement appear to need further discussion. The first area of 
disagreement is the timing of a requirement for an undergraduate degree. Specifically, is 
a degree required before entrance, or before certification, and under what circumstances 
should the requirement be waived? The second area concerns whether the degree should 
be associated with a business discipline. The third area mixes the issues of advanced 
formal education and multiple levels of certification. We’ll reserve this discussion for a 
later cycle. I developed the following questions to help us further explore the issues of 
timing, exceptions, and discipline. Please answer the questions using as much space as 
you need to support your position. 

Questions : 

1. Why is it important for the individual to possess an undergraduate degree before 
being granted entrance into the contracting career field? 

2. Why is it important for the individual to possess an undergraduate degree before 
being granted professional certification? 

3. Under what circumstances would a formal education requirement be waived? 

4. What value does formal education in a business related discipline provide to the 
workforce that cannot be achieved through other means (e.g., formal education in 
other disciplines, experience or training)? 


End Cycle 2a . 
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Appendix G. Synopsis of Responses For Delphi Cycle 2a 


Table 22. Civilian Responses to Delphi Cycle 2a / Question 1 

Why is it important for the individual to possess an undergraduate degree before 
being granted entrance into the contracting career field? 6 of 6panelists responding 

I don’t believe it is necessary to have a completed degree before entrance at the 5 pot 9 
level. However, any advancement beyond that should require a degree. 

Recent news: Because this year’s DoD Appropriations bill requires it [ respondent 
included a full text excerpt from Section 1724 of title 10, United States Code, as amended 
by Public Law 106-398 (Section 808)]. Attaining a bachelor’s degree indicates the 
candidate has shown the ability to accomplish a significant long-term goal and prioritize 
their actions to accomplish this goal. The individual acquires a body of knowledge 
through the process of attaining the degree. While everything learned in this process may 
not relate to knowledge required to perform on the job, the individual develops basic 
skills in acquiring the degree. The degree is evidence the candidate has some skill in 
researching, organizing, analyzing and communicating (orally and in writing). The 
bachelor’s degree has replaced the high school degree as an indicator the candidate 
possesses the expected entry-level qualification skill set in American society. If we are 
not recruiting from the American population that has acquired bachelor’s degrees, we are 
recruiting people from the lower half of the available workforce pool. 

It depends at what level the person comes into the career field. If they enter at the GS 7 
or 9 positions the 24 business credits are enough to get them started but a degree needs to 
be obtained for the GS 11 and above. Any targeted growth position of a GS 11 or above 
should require the degree before appointment. 

I don’t think it should be required before entrance into the program, as long as it is clear 
it will be required before certification and they are actively pursing their degree. 

By and large the private sector counterparts a government contracting professional will 
be dealing with will have degrees. A degree is necessary just to establish a level playing 
field. The Air Force has used the COPPER CAP program to be extremely selective with 
new civilian accessions. 

We have been on a long path to "professionalize" the contracting career field. The degree 
requirement is one method to take us further on that journey so that contracting has the 
same regard in the acquisition business as does, say, engineering. I also personally think 
that a college graduate has a far better opportunity for advancement within the career 
field than the non-grad. Among civilians, it is literally impossible to progress above the 
GS-11 level at most locations without a degree. It takes a Masters at many locations to 
progress above the GS-12 level. 
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Table 23. Military Responses to Delphi Cycle 2a / Question 1 

Why is it important for the individual to possess an undergraduate degree before 
being granted entrance into the contracting career field? 5of6 panelists responding 

How can we ever be considered a professional career field without a degree requirement? 
We assume that [lawyers, engineers, and accountants] have a minimum educational 
requirement that is further supplemented through professional certifications and 
experience. Why would we ever consider anything less for our profession? We should 
continue to raise our standards and require more stringent requirements to enter and stay 
in the contracting career field. To me, requiring a degree is the very foundation of the 
career field. Anything less reduces our stature. The one exception is for our enlisted. 
Their primary purpose is for operational contracting and to be prepared to deploy. 
Training to deploy should be their primary focus. But even they should be strongly urged 
to obtain a degree. 

I don’t think having a degree is an absolute must for entrance. In operational contracting, 
50% of workforce is enlisted. The overwhelming majority enter the career field as a 
cross trainee without a degree. The AF couldn’t maintain contingency contracting 
support for the warfighter without enlisted contracting members. Officers entering the 
career field will always have undergrad degree. Civilians entering contracting should 
have minimum of 24 business hours of coursework. To set an undergrad degree as 
requirement would hamstring the ability of most operational wings to maintain effective 
contracting support. 

I don’t think it is important. In order to progress up the ranks, I think an undergraduate 
degree should play a significant factor. 

Part of being a profession is maintaining the public image that special knowledge, skills 
and abilities are required to perform the tasks required of the professional, and that these 
can only be obtained through formal education followed by experience.. .Part of the 
reason for a degree requirement prior to entrance is to create barriers that help prevent 
erosion of the profession. Think about what would happen if you allowed someone 
entrance to the field without the formal education requirement being completed, and they 
learned all the necessary knowledge, and developed the required skills such that they 
were able to perform all the functions. Their ability to perform the job without the 
personal sacrifice would encourage others to do the same. 

If we ever want to become a professional series, we must have an educational entrance 
standard.. .contracting folks are being challenged with much more than issuing an order 
or being a file clerk. The Business Advisor role, though not formalized in our standard 
job descriptions, is a real world event. From this perspective, it is imperative that we 
have a workforce that is educated, especially in business related subjects. Once hired, 
few people take or have the time to attain an undergraduate degree—these folks are at a 
disadvantage from the start and typically become disgruntled employees as years pass 
and promotion opportunities never come. 
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Table 24. Civilian Responses to Delphi Cycle 2a / Question 2 

Considering APDP and other contracting professional certifications (CPCM, 
CACM...), why is it important for the individual to possess an undergraduate 
degree before being granted professional certification? 6 o£6_panelists responding^ 

I think it is important to differentiate between the APDP “box top” certifications (just 
send in your box tops in the form of completed courses and you are certified for life) and 
the NCMA professional certifications, which require rigorous examinations and periodic 
re-certification. An undergraduate degree is a reasonable prerequisite for certification in 
a quasi-professional field such as contracting. 

Certification demonstrates a level of professional development. 

The NCMA CACM does allow for certification prior to bachelor degree (or at least it did 
when I took it). The CPCM does not because it assumes a professional contracts person 
has the academic credentials. In the business world, many contract folks are lawyers or 
at least have their Masters. 

The value of professional certification is its credibility as an indicator the holder has met 
some defined criteria. The more rigorous the defined criteria, the more esteem and 
credibility (and thus value) the certification will hold. The concept of professionalism 
indicates the “professional” has demonstrated a level of proficiency in a defined body of 
knowledge. Without a degree requirement, it will be hard to demonstrate the professional 
certification has any more value than wallpaper. 

Because it is called a professional certification. 

I think we're talking about " professional building blocks" in this case. The foundation is 
formal education. In the same way that we certify physicians, dentists, and lawyers, we 
require a specific level of formal education from accredited sources before we grant 
certification. To do this in reverse, cheapens or minimizes the value of the certification 
in my mind. In the worst case it renders the certification to a meaningless status. If we 
didn't believe in the professional nature of the work we could talk in terms of 
apprenticeships and journeymen and trade associations and the sort of certification the 
trades grant. 
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Table 25. Military Responses to Delphi Cycle 2a / Question 2 

Why is it important for the individual to possess an undergraduate degree before 
being granted professional certification? 5_o[6_panelists responding _ 

Under the current APDP 3 tiered process, an undergrad degree shouldn’t be required for 
level one certification. The workforce filling entry/lower pay grade positions will not 
have a degree. I can see the possibility an undergrad degree requirement for level 2 and 
MS for level 3. However, much more study on the affect this would have on workforce 
will have to be accomplished. 

Based on [my answer to Question 1], everyone in career field should have a degree. 
Therefore it has to be a requirement for certification. A certification should be built on 
the degree by requiring certain specific training and experience. Certification should 
show a person has gone through a stringent program to obtain the certification. A 
certification should say a great deal about the person and their capabilities. 

Your formal education is an important part of progressing in contracting. Professional 
certification shows a level of mastery of the technical aspects of contracting. An 
undergraduate degree is one way of evaluating the technical mastering of the contracting 
career field. Therefore, a certain portion of an acceptable undergraduate degree should 
be in business. 

If we are serious about professional certification, then it is imperative that an education 
standard be part of the certification process. 

One respondent stated the answer was the same as that given for Question 1. 
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Table 26. Civilian Responses to Delphi Cycle 2a / Question 3 


Under what circumstances would a formal education requirement be waived? 

6_of6 panelist.s responding _ 

A long-term (ten years or longer) history of performance demonstrating the individual 
advanced from performing the simplest career functions to those expected of a 
professional. This would be an apprentice to journeyman progression that can be 
demonstrated by assignments and accomplishments over an extended time period. 
Demonstration of performance at the managerial level of complexity within the career 
field. For example, an individual may move from one career field to another and 
demonstrate performance at the managerial level of the second career field through an 
adroit use of managerial skills, which make up for technical shortcomings, (anecdotal 
explanation... I’ve worked for some General Officers who demonstrate an exceptional 
understanding of contracting concepts and the acquisition process even though they have 
not spent one minute of time in a contracting position. They have however demonstrated 
evidence of contracting knowledge that far surpasses our average “professional.” On the 
other hand, I’ve worked for a couple that didn’t have a clue what the PCO was talking 
about when the subject matter was contracting.) 

For certification? If the certification did not have “professional” in the title. 

For entry into the program, when an individual is actively pursuing completion of their 
undergraduate degree. 

In the future, they should not be waived, except for the few remaining employees who 
met the original “10 years of experience as of 1991” criterion. 

I do not see any circumstance where is should be waived at the Bachelor Degree level. 

Only in the case of enlisted members. We in the AF are currently working a class waiver 
to Section 808 of this year's authorization bill to specifically exempt our enlisted troops, 
who stay within the career field to fill Contingency Contracting obligations only; a very 
limited segment of the total contracting force. 
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Table 27. Military Responses to Delphi Cycle 2a / Question 3 

Under what circumstances would a formal education requirement be waived? 

5_of6_ panelists responding _ 

Formal education should be waived only in circumstances where the individual has 
extensive experience. However, the waiver should only be for a period long enough to 
give the individual time to finish their formal education. 

For entrance in the career field, none. For professional certification, some equivalent 
level of experience or significant completion of undergraduate business courses, maybe. 

I don’t believe a degree should be a requirement for entry into the career field or initial 
APDP certification. 

At least one person made an argument for enlisted personnel exception. Many 
professions require formal degrees for entrance into certain positions, but not others. For 
example, in the medical profession, nurses and physician assistants need undergraduate 
degrees, and doctors require graduate degrees. Other positions like nursing assistants and 
technicians require training, but no formal degree requirement. A similar case can be 
made in the law profession. In both fields, the supporting personnel without degree 
requirements are perfomiing functions essential to the profession, but they are excluded 
from performing specific functions reserved for the “professionals”. These people are 
highly qualified, and normally receive extensive training, but they are not considered 
“professionals”. These “paraprofessionals” are often certified, but it’s a different 
certification system. Hierarchy of positions is very important in professions. There are 
contracting managers, contracting officers, buyers and administrative support positions 
on the systems acquisition side of contracting. Certain authority is reserved for those 
personnel who hold a warrant. They are probably the “professionals”. 

Waive the requirement only for enlisted. 
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Table 28. Civilian Responses to Delphi Cycle 2a / Question 4 


Does your answer to Question 1 apply equally to officer, enlisted, and civilian 
personnel? Why or why not? 6 o/(5 panelist^ responding^ _ 

Equally to all, because the level of certification demonstrates a level of professional 
development, not position held. 

It does not apply to enlisted personnel grades E-l through E-6. The types of positions 
these individuals would be required to hold are technical and administrative in nature, not 
requiring a baccalaureate degree. They perform some very important functions, not the 
least of which is carrying the lion’s share of the contingency contracting duties. It is 
unlikely the services could recruit a sufficient number of degreed individuals to fill all 
these billets. I could see requiring a degree for E7-E9s who perform the superintendent 
duties, but I think it would conflict with overall enlisted promotion program. Officers 
and civilians all start on career ladders that could lead to senior acquisition positions, so 
formal education is a must. 

Yes it would apply to officer, enlisted and civilian personnel. I envision a merit based 
career field wherein individual are judged by their knowledge and accomplishments, not 
by the method they entered the work force. 

Yes, because the career field is the same and all should be treated equal. I still think the 
Air Force is going to have to change the thinking on for enlisted Contingency Contracting 
Officers and the required certifications required. 

Officers and civilians, yes. Enlisted - perhaps not because they are not required to have a 
degree for advancement. At least they were not in the past. 

No, only officer and civilian. 
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Table 29. Military Responses to Delphi Cycle 2a / Question 4 

What value does formal education in a business related discipline provide to the 
workforce that cannot be achieved through other means (e.g., formal education in 
other disciplines, experience or training)? 5_of_6_panelist^ responding _ 

The business world is broken into a variety of different disciplines (logistics, acquisition, 
manufacturing, accounting, etc.). A business degree normally gives you the breadth of 
knowledge in all the different disciplines. You can normally get this knowledge from 
other sources; however, the business degree will provide you with a better foundation for 
learning. 

If we just look at our business processes today the answer would be “moderate” value. 
But, looking beyond today and into the future world of enterprise business environments, 

I believe business degrees will offer “significant” value, especially as business schools 
adapt their curriculum to teach advance trends in e-commerce and IT solutions. 

For an initial background, 24 hours of business related coursework is a good start for 
entry-level civilians. Enlisted should be waived from this requirement. However, as 
stated previously, a business degree isn’t the end all. We need professionals who can 
think logically, communicate clearly, and understand how to work in a team environment 
to accomplish tasks. These skills can be developed within the confines of any accredited 
degree program. The specific DOD acquisition specialty training will be provided as a 
matter of course. 

I maintain that a business degree or related discipline is essential. This degree covers the 
types of knowledge and skills that are required for the contracting professional. Again, 
how can we manage, monitor, and negotiate with a contractor without having a basic 
understanding of how a business operates? A business degree helps with so many vital 
areas. This includes: accounting and all that is related such as overhead rates; business 
communication, which is essential for the contracting professional by sharpening both 
writing and oral communication skills; and [it] helps with leadership and management 
skills.. .The contractors we work with certainly require these skills. We should not send a 
less educated workforce to deal with them. I take strong exception that a psychology or 
sociology degree is just as good. These degrees are a joke. They are for football players 
to major in. Why we would want to lower our standards? Lets get serious about 
developing contracting as a profession. This means requiring tough standards that are 
relevant to the service we perform. Anything less and we are wasting our time. 

One respondent stated he did not know of any particular value furnished by a business 
degree over other disciplines. 
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Appendix H. Delphi - Cycle 2(b) - Training (Civilian) 


The training requirement of most certification programs is accomplished through 
the completion of PCE courses. What value, if any, does this type of training offer 
beyond that received in the workplace? What specific courses or subject areas 
should be required? 

I’ve consolidated your responses to the question above. The number in the parentheses 
following each statement indicates the number of you who made similar responses; some 
respondents may be represented by multiple responses. There was general agreement to 
the value of classroom training, but some respondents seemed dissatisfied with the 
particular training being offered or the method by which it is received. In this cycle, we 
need to make sure we have consensus on the areas of agreement, and then further explore 
areas where there is dissimilarity in opinions. 

The first section contains points of relative agreement. I want to make sure we have 
consensus on these points. Please comment on the extent you agree or disagree with each 
statement. Provide your reason(s) for disagreement. 

Areas Of Agreement : All respondents believe that classroom training provides value to 
the workforce. The support offered for this assertion included: 

- PCE develops needed skills.. .training in the mechanics of the trade. (2) 

- PCE exposes an individual to contracting methods and ideas they might not be 
ordinarily exposed to. ..allowing for a broader academic theoretical discussion. (2) 

- PCE reinforces good on-the-job training. 

- PCE offers the opportunity to network with peers and draw on each other’s 
experiences. (2) 

- PCE catches current trends.. .is the best way to get new ideas out to the field. (2) 

Areas for Further Discussion : At this point, the group opinion on specific courses or 
topics to require is indeterminable. 2 of 6 respondents did not indicate specific courses to 
require but did raise another interesting issue that merits discussion. 

- 1 of 6 respondents required the following courses or topics: Contract Law, 

Contract Administration, Pricing, Negotiation Techniques, and Market Research. 

- 1 of 6 respondents added the requirement of teaching Quantitative Methods. 
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- 1 of 6 respondents thought broader training should be added, “we need to have 
training focused on other disciplines (finance, history, etc.) to help the workforce see the 
big picture.” 

- 2 of 6 respondents questioned whether certification programs have heightened the 
need for PCE, “prior to the APDP certifications in contracting, virtually 100% of our 
workforce had completed more than the requisite courses...certification requirements 
have had little to no impact on the PCE training or workforce receives,” and “[once an 
individual is certified] the availability of government sponsored courses is very 
limited., .it forces people to seek out organizations, such as NCMA, who can provide 
these courses.” 

- 1 of 6 respondents stated that, “if [PCE] is received before on-the-job training, the 
individual may have difficulty in finding the correct frame of reference for the topics 
being taught.” 

Questions : 

1. To what extent do you agree with requiring the courses listed above? 


2. Do the current certification programs satisfy these requirements? 


3. To what extent has establishing certification programs emphasized PCE for the 
workforce? Is this favorable? 


4. What experience should be required before attending PCE? 


End Cycle 2b . 
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Appendix I. Delphi - Cycle 2(b) - Training (Military) 


The training requirement of most certification programs is accomplished through 
the completion of PCE courses. What value, if any, does this type of training offer 
beyond that received in the workplace? What specific courses or subject areas 
should be required? 

I’ve consolidated your responses to the question above. The number in the parentheses 
following each statement indicates the number of you who made similar responses; some 
respondents may be represented by multiple responses. There was general agreement to 
the value of classroom training, but some respondents seemed dissatisfied with the 
particular training being offered or the method by which it is received. In this cycle, we 
need to make sure we have consensus on the areas of agreement, and then further explore 
areas where there is dissimilarity in opinions. 

The first section contains points of relative agreement. I want to make sure we have 
consensus on these points. Please comment on the extent you agree or disagree with each 
statement. Provide your reason(s) for disagreement. 

Areas Of Agreement : All respondents believe that classroom training provides value to 
the workforce. The support offered for this assertion included: 

- PCE courses allow students to focus completely on a given subject without the 
interference of other duties. (2) 

- PCE exposes an individual to contracting methods and ideas they might not get 
exposed to during routine OJT.. .offers a wide range of scenarios and explores different 
options to resolve issues. (3) 

- PCE allows people to acquire specific knowledge and skills...important for those 
entering the field to learn the tools necessary for the profession. 

- PCE offers the opportunity to network with peers and draw on each other’s 
experiences. (6) 

Areas for Further Discussion : Many of the respondents stated that the current PCE 
curriculum is satisfactory. One contrasting opinion looks at a unique issue that merits 
further group discussion. 

- 2 of 6 respondents did not indicate specific courses to require. 
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- 2 of 6 respondents believe the current curriculum meets the necessary requirements, 
“the basic courses as currently structured offer the desired curriculum,” and “it’s a good 
mixture of general contracting to specific contracting issues.” 

- 1 of 6 respondents listed the following course work: Source Selection/Best Value, 
Contract Management, Contract Law, Services, Construction, Systems, and Contingency 
Contracting, Pricing, and Electronic Contracting. 

-1 of 6 respondents feared that, in terms of PCE, the contracting community is 
serving the certification programs, rather than the certification programs servicing the 
workforce, “I’m not sure the majority of training is still accomplished by sending people 
to PCE.. .1 believe we should have the courses we had ten years ago.. .pricing was done 
in three phases, each [one] difficult, but people learned to price.. .we had specific courses 
on services contracts.. .we used to send everyone to contract law.. .we had three courses 
in how a major program was conceived and brought through the milestones.. .now we 
just tell people to sit in their cubical and do [computer-based training] and ‘oh, by the 
way, answer your phone while you’re there’.” 


Questions : 

1. To what extent do you agree with requiring the courses listed above? 


2. Do the current certification programs satisfy these requirements? 


3. How is the workforce benefiting from the level of difficulty in current course 
offerings? 


4. Why should certification programs require in-residence PCE, rather than giving 
credit for on-line or software instruction? 


End Cycle 2b . 


125 



Appendix J. Synopsis of Responses For Delphi Cycle 2b 


Table 30. Civilian Responses to Delphi Cycle 2b / Question 1 

To what extent do you agree with requiring the courses listed above? 

5of6 panelists responding ___ 

I agree with all of them. 

Concur with this listing of required courses: Contract Law, Contract Administration, 
Pricing, Negotiation Techniques, and Market Research. I also concur that the 
quantitative methods class should be required for Level III certification. 

I think the idea of broader training is one that should be addressed in the Bachelor Degree 
requirements. Having just a Business degree may not be the only useful one for this 
career field. I do believe that all contracting personnel need training in people skills. 

Most academic courses described above do not address this important aspect if the career 
field. Negotiating skills, meeting skills, personal interaction and organizational behavior 
are very important courses that need to be mandatory for anyone at the GS 12 level and 
above. 

I agree with Contract Law, Contract Administration, and Pricing & Negotiation 
Techniques. In addition, I like the idea of a broader training focus by adding the 
possibility of requiring an elective type course to be taken from another field. There is 
the need for a basic course, such as CONS 101. Market Research is included in CONS 
101 and CONS 202, and a separate course isn’t required, although possibly enhancing 
what is currently taught might be helpful. 

Some, but not all would benefit from quantitative methods, (many would have had that 
discipline included in their undergraduate work others will be in jobs that never require 
it). Anyone who is going to spend any time as a contract negotiator needs to have some 
Cost Accounting to supplement the Pricing courses. 
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Table 31. Military Responses to Delphi Cycle 2b / Question 1 


To what extent do you agree with requiring the courses listed above? 

3 of 6 panelists responding^ _ 

All the courses listed above should be required. Getting certified should be a demanding 
process. Once certified, it should show that a person is a well-rounded, well-educated 
contracting professional. 

Strongly agree. The focused discussions on these topics, provided in a classroom setting, 
provides an exchange of information which prepares the student for hundreds of real 
world events that they would not otherwise experience. For example, a company goes 
bankrupt.. .the book solution is pretty straightforward but the difficulty comes in 
implementing that solution. A classroom discussion is the only way to explore that issue. 
Ultimately, I’d like to see courses on Contract Law, Services Contracts, Public/Private 
contracting issues, Contract Types, Technical / Engineering contracts, Major programs 
(both basic & advanced), Deployed contracting issues, commercial contracting, and 
contracting innovations, and contracting system(s) (i.e. SPS [or for the next few years: 
MOCAS, DIRAMS, BCAS, etc.). 

You should have the following courses: 

1. 1 to 3 basic courses that take you from cradle to grave on an acquisition. This is 
important because you have to know the basics before you can think outside the box. 

2. Pricing - Understanding how a proposal is developed and priced is so important in 
evaluating a proposal and should be one of the fundamental courses every contracting 
professional should take. 

3. Law - Another essential training course. This course should be case based and 
require critical thinking. This should not be to give basic law terms and definitions. 
That’s why we have the basic courses. 

4. Contract Administration - This should focus on how to administer a contract and 
should also be case based and focus on critical thinking, dispute resolution. 

5. Specialized training - This should include instruction in systems, contingency, 
services, construction, electronic contracting, etc. This should supplement your basic 
contracting training. 
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Table 32. Civilian Responses to Delphi Cycle 2b / Question 2 


Do the current certification programs satisfy these requirements? 

5 of 6 panelists responding^ __ 

Yes, I believe they do. 

The current certification process encompasses needed courses, with the exception of 
elective type courses. 

No. 

No these skills are sorely lacking in the current programs. 

I hope I’m not getting confused here. This question says “programs” and I thought we 
were just talking about APDP. Assuming that we are addressing just APDP, this program 
addresses the basic needs but should always be subject to review and further tweaking. 


Table 33. Military Responses to Delphi Cycle 2b / Question 2 

Why is it important for the individual to possess an undergraduate degree before 
being granted entrance into the contracting career field? 3 o/(5 panelists, responding 

Do the current certification programs satisfy these requirements? 

No, current cert programs are, by in large, square fillers and only provide minim 
exposure to issues. In some respects we had a better-trained workforce before the 
certification programs, our problem was that it wasn’t evenly spread. Some people were 
trained very well, others not at all. Now we’ve improved the minimum but have fewer 
very well trained people. 

I believe the certification process should be more demanding. As discussed before, it 
should have strict academic and training requirements along with years of the right 
experience included. 
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Table 34. Civilian Responses to Delphi Cycle 2b / Question 3 

To what extent has establishing certification programs emphasized PCE for the 
workforce? Is this favorable? 5_of6_ panelists responding^ _ 

It is favorable but does not go far enough. It has forced the workforce to take minimal 
requirements. Once they have the APDP level certification many experienced people do 
not seek out additional training opportunities. There is no re-certification courses 
required unlike the NCMA certification programs. 

The mandatory APDP makes for a much more orderly acquisition of contracting 
knowledge than was previously possible when availability of courses and opportunities to 
attend seemed to be almost random. I believe that the average 1102 with five years 
experience today is far ahead of where I was at the same point in my career in the 1970s. 

It has increased the emphasis and importance of attending the PCE courses by a small 
amount. Yes it has been favorable, although not significant. 

The APDP certification program institutionalized the training need and actually started 
emphasizing the need to train civilians better. It forced management to insure everyone 
in the field received training. It has been very favorable. 

One respondent indicated he or she had no opinion for this question. 
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Table 35. Military Responses to Delphi Cycle 2b / Question 3 


How is the workforce benefiting from the level of difficulty in current course 
offerings? 3_of6 panelists^ responding^ _ 

They all pass if they show up and breathe. Has anyone every flunked? Probably not. 
There are to many job related issues if they do. We do the DoD a disservice by 
perpetuating less than excellence as acceptable. So, to address that we’ve now got a law 
that says everyone must have a college degree. Wrong answer. 

I’m not sure how the courses are being conducted lately. One of the problems with 
training in the Air Force is that training is now controlled by AETC. AETC does a good 
job with pilot training and tech training by not so good with academic type training. The 
instructors are not given the freedom to make the courses top quality. The focus is on 
throughput, not quality training. In addition, for someone to leave acquisition to go 
become an instructor in AETC for three years will most likely hurt his or her careers. 
This makes it tough to get quality folks teaching our contracting professionals. 

I think the current courses are way too easy. I have yet to be challenged in a contracting 
course. What happens is you get people that are APDP Level II or III certified and don’t 
know much about contracting because the courses are so easy. I like NCMA’s CPCM 
and CACM tests. Course completion should not be enough. A final test should be 
required to truly test an individual’s knowledge of contracting. 
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Table 36. Civilian Responses to Delphi Cycle 2b / Question 4 


What experience should be required before attending PCE? 

5_of_6_panelists responding _ 

Require 6 to 8 months experience on the job before attending any course. Most attendees 
tell me they learn so much more when they’ve been exposed on the job first. In addition, 
requiring a graduated increase of experience for higher-level courses like the existing 
process is appropriate. 

Minimum of 6 months. 

I believe current APDP level required experience is too short. A minimum of 2 years for 
Level I, 6 years for Level II and 10 years for Level III. Running trainees through these 
programs too quick and giving them their certifications without the experience just makes 
industry question the validity of the certification process in the federal government. 

A minimum of 3 months on the job before attending the basic contracting course would 
be advisable. 

Recommend at least two months before attending CON 101. After that, 3-4 months 
experience for the remainder of the classes required for Level I certification should be 
adequate. Recommend at least one full year’s experience before taking Level II required 
courses and at least 3 year’s experience before taking courses required for Level III 
requirements. 
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Table 37. Military Responses to Delphi Cycle 2b / Question 4 


Why should certification programs require in-residence PCE, rather than giving 
credit for on-line or software instruction? 4 of 6panelists responding _ 

All six participants identified the importance that PCE provide an opportunity to network 
with peers and draw on each other’s experiences. This can only occur in an interactive 
group setting (residence, traveling instructor, or interactive video). I think we need to 
make a determination of what content can be taught through safe-paced means (e.g., 
computer based training) and what content should be reserved for the group. One 
approach would be to require people to successfully complete a CBT portion prior to 
attending a group session. This could have the dual benefits of increasing the quality of 
group interactions by preparing people with a common vocabulary prior to class, as well 
as decreasing the time required for group interactions. A detailed analysis is beyond the 
scope of this exercise, but discussing some of the ideas may prove to be an important 
start. 

Let’s really think about on-line and software instruction. The student is pretty much by 
him/herself and trying to do the work at the day-to-day work place. That person doesn’t 
have anyone to discuss the issues with nor ask questions of. There is no exchange of 
ideas or experiences. Learning occurs best in the discussion and interaction with fellow 
students and instructors. My experience is that the in-class text and test is a minimum, 
the instruction is significantly broader than the text. The in-class questions focus on 
where the student is interested and in need of help. The latest issues are quickly 
incorporated in the curriculum. Overall the learning experience is significantly better. 

Interaction with the other students is essential. Otherwise, training becomes a square¬ 
filling exercise. We can certainly save time and money by doing on-line courses, and 
conducting classes by VTCs. However, if we are really serious about training we need 
to bring contracting professionals into a training environment where they can network 
and share ideas from their diverse backgrounds. I also think it’s important for students to 
concentrate solely on school while taking a course and not be distracted by work in any 
way. Just like officers taking SOS. Taking it correspondence just does not measure up to 
the experience of attending in person. We have to be dedicated to the importance of 
training, and contracting professionals will have to make arrangements to attend training 
as needed. 

I think many of the basic contracting courses can be taken on-line. If the validation of the 
student’s ability is to take a multiple-choice test, then there are ways of taking a closed 
book test on-line. This should be more than acceptable. However, I believe the higher 
level, more specialized training should be case study and tested through written, essay 
exams. The specialized training courses would need to be more free flowing because of 
the case study/critical thinking aspect and an on-line course would not be able to 
accommodate. 
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Appendix K. Delphi - Cycle 2(c) - Experience (Civilian) 


What is the value of requiring a minimum amount of c ontractine experience before 
granting certification? What is the appropriate way to measure that experience? 
What specific tasks or areas of experience should be required? 

I’ve consolidated your responses to the question above. The number in the parentheses 
following each statement indicates the number of you who made similar responses; some 
respondents may be represented by multiple responses. There was general agreement to 
the value of experience, but there is some disagreement as to how the experience should 
be measured. In this cycle, we need to make sure we have consensus on the areas of 
agreement, and then further explore areas where there is dissimilarity in opinions. 

The first section contains points of relative agreement. I want to make sure we have 
consensus on these points. Please comment on the extent you agree or disagree with each 
statement. Provide your reason(s) for disagreement. 

Areas Of Agreement : All respondents believe that experience provides value to a 
certification. The support offered for this assertion included: 

- Without performing the task, mastery will never be achieved. 

- Education and training are only effective when put into practice.. .most contracting 
knowledge comes from the job, not the classroom. (2) 

- Experience provides the ability to screen distracting and irrelevant information to 
discover the root of the problem.. .experience also involves dealing with political and 
emotional issues not always present in straightforward classroom exercises. 

- Requiring experience before certification allows the assumption of a level of 
technical competency, unlike academic degree programs. 


Areas for Further Discussion : 4 of 6 respondents submitted a particular method of 
measuring experience before granting certification. At this point it is not possible to 
determine the overall group opinion of the best measure of experience. 

-1 of 6 respondents stated the current method of measurement is appropriate. 
Currently, professional certification programs require a number of years experience in 
contracting or a contracting related field. 
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- 1 of 6 respondents stated that “x number of years in a certain job could be a way of 
measurement.” This response is similar to the first but seems to require experience in 
specific areas of contracting rather than just a minimum time in the career field. 

- 1 of 6 respondents stated that the current measure is appropriate but the level of 
experience required is insufficient, “experience time should equal that of the complexity 
of courses required for that certification level.. .a minimum of 2 years experience for 
Level 1.. .6 years for Level 2 and 10 years for Level 3 would be more in line with the 
course material.” Note: The levels referred to are those of the APDP certification 
program and don’t apply to other certifications .” 

-1 of 6 respondents added that the individual should meet the requirements 
necessary for obtaining a PCO or ACO warrant before being granted certification. 


Questions : 

The APDP certification program has the following experience requirements: 1 year 
in contracting (Level I), 2 years in contracting (Level II), 4 years in contracting 
(Level III). The NCMA CPCM requires 2 years experience in: procurement, legal, 
MIS/IT, inventory mgt, project mgt, production, R&D, marketing, QA, finance, or 
business mgt. 

1. Should the experience requirements insist on experience in more than one area of 
contracting? 


2. What improvements can be made to these requirements? 


End Cycle 2(e) . 
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Appendix L. Delphi - Cycle 2(c) - Experience (Military) 


What is the value of requiring a minimum amount of c ontracting experience before 
granting certification? What is the appropriate way to measure that experience? 
What specific tasks or areas of experience should be required? 

Eve consolidated your responses to the question above. The number in the parentheses 
following each statement indicates the number of you who made similar responses; some 
respondents may be represented by multiple responses. There was general agreement to 
the value of experience, but there is some disagreement as to how the experience should 
be measured. In this cycle, we need to make sure we have consensus on the areas of 
agreement, and then further explore areas where there is dissimilarity in opinions. 

The first section contains points of relative agreement. I want to make sure we have 
consensus on these points. Please comment on the extent you agree or disagree with each 
statement. Provide your reason(s) for disagreement. 

Areas Of Agreement : All respondents believe that experience provides value to a 
certification. The support offered for this assertion included: 

- Certification denotes a level of professionalism and competency.. .standards for 
experience must be set to ensure the integrity of the certification process. 

- The knowledge and skills obtained through classroom training can only be matured 
through application to actual contracting processes.. .no amount of education is sufficient 
without hands-on contracting experience. (4) 

All respondents also agreed that a broad base of experience is desirable, although 3 
respondents neglected to list specific areas of experience. 

Areas for Further Discussion : The issue of which specific areas of experience to 
require may be explored further in a future cycle. At this point, the most contrast 
between respondents is seen in the issue of the proper way to measure experience. 

- 1 of 6 respondents did not name a preference for measurement, “the only way to 
ensure an individual has mastered the necessary skills to perform all job requirements is 
to give that individual work in every area of the business.. .this is not practical.. .the 
default is to grant certification after a minimum amount of time on the job.. .this is by no 
means an optimal solution, either.” 

- 1 of 6 respondents indicated an individual’s experience should be measured by the 
number of years on the job, “I don’t think it appropriate to segregate by type of 
contracting.. .how do you classify the experience... some contracting in ASC is more like 
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base contracting and some at many HQs is more like systems. ..just go with the number 
of years.” 

- 4 of 6 respondents believe the measurement should be more task specific, “the 
contracting position can be broken down into a number of essential tasks... tasks can then 
vary in terms of their relative effect on organizations (e.g., criticality, dollar value, 
priority).. .experience should be measured in terms of the quantity of essential tasks 
performed of varying quality,” and “experience should be measured by time spent in a 
specific job... years spent in a SPO vs. operational contracting vs. administration... to rise 
to the senior levels in contracting a person should have a background in [all three areas],” 
and the enlisted OJT skills upgrade is the best way I’ve seen.. .a list of core tasks that any 
contracting professional would need to know.. .the system is susceptible to pencil 
whipping; however, you have to rely on the integrity in the system and trust the 
individuals certifying an individual,” and “a base of knowledge must be obtained for each 
level of certification... [basic level certificants] must understand and have accomplished 1 
year of simplified acquisition procedures. ..[intermediate level certificants] must 
demonstrate understanding and 3-4 years experience in services, construction, or systems 
contracting... [advanced level certification] should require 8 years of above experience.” 


Questions : 

The APDP certification program has the following experience requirements: 1 year 
in contracting (Level I), 2 years in contracting (Level II), 4 years in contracting 
(Level III). The NCMA CPCM requires 2 years experience in: procurement, legal, 
MIS/IT, inventory mgt, project mgt, production, R&D, marketing, QA, finance, or 
business mgt. 

1. Do these requirements support the group consensus that depth of experience is 
important? 


2. What improvements can be made to these requirements? 


End Cycle 2tct . 
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Appendix M. Synopsis of Responses For Delphi Cycle 2c 


Table 38. Civilian Responses to Delphi Cycle 2c / Question 1 

Should the experience requirements insist on experience in more than one area of 
contracting? 4 of 6panelists, responding _ 

I do not think experience in more than one flavor of contracting is necessary for 
certification. Always a help but not a pre-requisite. 

That might work in the Military workforce but with civilians it would be difficult. The 
new Standard Core Position Description (SCPDs) which are mandatory for the base level 
contracting offices now make most everyone a contract specialist who can do any tasks - 
pre or post award. There are few career broadening positions for civilians and because 
that is true they are quite competitive. We do not want to make it so hard that none of 
our career folks can get certified. 

The current APDP experience requirements are appropriate. I wouldn’t agree with the 
approach of requiring experience in more than one area, although putting it forth as a 
stated desire would yield the same overall benefit. Bottom line in the hiring process the 
most qualified individual with all areas considered gets the job, and the individual that 
puts forth more effort will in the end reap the rewards. 

To answer this effectively, we need to consider what the contracting officer of the future 
will be. Some say there will be a blending of program management with contracting. If 
so, then experience in the corollary field would be necessary. 
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Table 39. Military Responses to Delphi Cycle 2c / Question 1 


Do these requirements support the group consensus that depth of experience is 
important? 3o£6_ panelist responding^ _ 

These standards are acceptable and do show that depth of experience is important. I may 
increase the number of years required, but it’s not necessary. It must be remembered that 
these standards are minimums. 

I’m not sure they do. These requirements simply require time in the career field. I’ve 
had many employees with “twenty years experience, one year at a time.” In other words, 
they had done the job for years and everything was still new to them. While I agree that 
depth of experience is what we’re all looking for, I doubt the ability to consistently 
measure that experience. “One person’s junk is another’s treasure” so to speak. The real 
question is “How do we train our people and how do we measure the quality of that 
training?” Another important point to remember, you have three separate and specific 
different training goals. Enlisted, officer and civilian all are in the career field but fill 
different roles. (Note that I lump civilian in with enlisted at the base level.). To 
understand what depth of experience is desired you need to understand what the person is 
being trained for. We need a conversation on that issue. I submit we train civilians for 
long-term functional expertise; enlisted for functional expertise and the ability to perform 
a combat support function; and we train officers not so much for functional expertise but 
for what I’ll call “strategic functional expertise.” 

That said, I do like the idea of an OJT record. Maybe we tailor that record for the 
specific training goals of each group. That was how it was done in the past. The 
requirement for officers was for the supervisor to complete the same OJT requirements as 
for the enlisted 5-level. In addition, we had to attend three or four classes. The 
requirement was to attend those courses within the first two years, Contract Law within 
the first year. The record was inspected during ORIs. 

No, but it may not be the right question. I look at practical decisions in day-to-day 
operations. Do I really care if a person filling a Level II position has 2,3,4 years of 
contracting experience? The answer is "it depends." I'm looking for someone to get the 
job done and in some cases the person with 2-years is the answer. In other words, I don't 
want some artificial certification requirement to handicap my ability to place people into 
jobs that I know they will excel in. On the other hand, depth of experience is important 
as you move up in grade and responsibility. So, maybe we need to rethink how we 
classify our positions and have flexibility in determining depth of experience within each 
level through the job referral process vice certification process. 
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Table 40. Civilian Responses to Delphi Cycle 2c / Question 2 

What improvements can be made to these requirements? 4_of6 panelists responding 

Extend the period required. The NCMA CPCM requirements have academic as well as 
experience requirements just to sit for the exam. You have to pass the exam also which is 
quite difficult. There are no final exams for APDP certifications levels. Maybe that 
would help. 

As stated above, include diversity as a stated desire. 

Define the role of the contracting officer in the post FASA environment and determine 
types of experience accordingly. 

Two years experience to test for a CPCM or CACM is insufficient. You can pass the test 
with study, but you're hardly a "professional" at that point. One size never fits all of 
course but I'd be more comfortable, if we have to use time in the field as a criterion, with 
four years. Why 4 years? That's the length of time we devote to training our "advanced 
copper caps", or interns, and bringing them to grade 11 or 12 (also called journeyman 
level by some). 
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Table 41. Military Responses to Delphi Cycle 2c / Question 2 

What improvements can be made to these requirements? 3 of6 panelists responding 

Add the OJT record. But, ensure a check is done of the individuals assigned organization 
by a HQ functional expert trained in OJT monitoring. Keep the time in career field as a 
requirement. The OJT record would have to include alternate methods of filling a 
requirement. (There may not be a major source selection in an organization for years.) 
For Officer’s I really believe that two tours in base level are important BEFORE making 
them squadron commanders. They really need to know the Air Force outside of the 
AFMC environment before they’re expected to interact with the shooters and enlisted 
culture. 

If you look at the Level III contracting jobs, they are significant positions requiring an in- 
depth knowledge of not only contracting, but also general leadership skills. Time and 
training (especially 4-years) do not necessarily insure an individual possesses necessary 
tools to successful perform the job. I believe requiring specific job experiences should 
also be considered based on individual positions and not generically. For example, the C- 
17 Director of Contracts job and the ASC/PK job are both Level III positions. I would 
argue that the ASC job requires more "experience" than the SPO job. The question is "do 
we address this issue in the certification process or in job referral process?" I would lean 
to the job referral process, because the local office controls that process. 

The best way to improve the standards is to require years of experience in specific areas. 

I believe this is the way the APDP certification program started. They required a 
minimum number of years of SPO experience. Many people where upset with this 
because they never worked in a SPO and probably never would. I think we should 
develop a matrix that shows what kind of experience would fulfill the requirements of 
earning certification. For example, instead of 2 years in a SPO, a person could meet the 
requirement by doing 4 years in some other area. However, we, as community, would 
probably never be able to come to gather to agree on what kind of experience is required 
to meet certification levels. 
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Appendix N. Delphi - Cycle 2(d) - Comprehensive Examination (Civilian) 


What is the value added by requiring the successful completion of a comprehensive 
examination before granting certification? 

I’ve consolidated your responses to the question above. 

There is no determinable group consensus at this point. Of 6 respondents, 4 were in favor 
of an examination requirement while 2 were not. At this point I would like to present the 
responses and allow you to consider the other arguments in answering the question at the 
end of this document. 

Responses : 

- As far as professional certifications - it would not be a professional certification 
without a comprehensive examination. For level one certification, I don’t believe a test 
in necessary - training and experience complete this. Level II is about the same. 

However, having a test for Level II and/or III may help in declaring contracting a 
professional field. 

- The primary benefit of a comprehensive examination is it gives credibility to the 
certification. If an individual has passed the comprehensive exam, then the individual 
has demonstrated they have a specified level of mastery of the material. If there is no 
exam, there is not confirmation that the individual has mastery of the material... only that 
the individual was in the classes or was at work for a specified period of time. For the 
certification to have credibility there must be a comprehensive exam and it must be 
possible to fail the exam. The professional certifications granted by AFIT in the past 
aren’t real prestigious. Everybody goes to the classes, everybody (well almost 
everybody) passes the classes and attaining the certification isn’t much of an 
accomplishment. The CPCM exam administered by NCMA on the other hand, had a 
60% fail rate in its early days. Successfully passing the CPCM is truly an 
accomplishment and is recognized as such by the professional workforce. 

- The requirement of a comprehensive exam forces the person to study areas, which 
they would not normally deal with on an everyday basis. The contracting professional 
should at least be exposed to all aspects of the field and the comprehensive exam does 
this. With contracting becoming a dynamic field one never knows where or what one 
will be doing next year. 

- If formal education, training, and experience are required, then examinations 
shouldn't be. Only consider examinations to substitute or waive formal education. 
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- There is little alternative if the goal is for the candidate to show mastery of the 
subject material 

- Overvalued in my opinion. We have study programs that tend to teach the test. 
Not optimal. 

Areas for Further Discussion : Please answer the following question. You are not 
confined to the context of one particular certification program; it is possible that none of 
the available certifications contain the optimal requirement. Concentrate on whether 
there should be a requirement for an examination, instead of whether that requirement is 
currently administered properly. Depending on whether the group is in favor of 
examination or not, we can discuss proper administration in a later cycle. 

1. In what way does requiring the completion of an examination strengthen, or fail 
to strengthen, a professional certification for the contracting workforce? 


End Cycle 2(dt . 
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Appendix O. Delphi - Cycle 2(d) - Comprehensive Examination (Military) 


What is the value added by requiring the successful completion of a comprehensive 
examination before granting certification? 

I’ve consolidated your responses to the question above. 

There is no determinable group consensus at this point. Of 6 respondents, 3 were in favor 
of an examination requirement while 3 were not. At this point I would like to present the 
responses and allow you to consider the other arguments in answering the question at the 
end of this document. 

Responses : 

- I am not in favor of a comprehensive examination for granting certification at this 
stage in the ADPD program. PCE test scores, OJT evaluation, appraisal feedback, etc., 
are adequate standards for determining whether or not an individual is able to perform the 
job. I would be willing to reconsider this position, if and when OPM changes the status 
of contracting to a professional series. 

- Comprehensive exams serve two purposes. The first and most obvious is reason is 
to assure a minimum standard for entry into the profession. The other second reason 
concerns increasing the level of commitment of the individual to the profession. One of 
the primary predictors of commitment to a course of action is the amount of effort 
required to initiate that course of action. This partly explains why hazing remains 
popular in fraternities and sororities, and boot camp is so effective. People who have 
struggled to complete a comprehensive examination will have greater commitment to the 
values and beliefs held by that professional field. 

- It demonstrates a minimum level of knowledge. It also shows dedication to the 
career. Just like accounting uses the CPA to distinguish between accountants. 

- I’m not certain the examination and study for the examination are discriminators. 
But, the demonstration of commitment shown by studying for and taking the examination 
is what I look for. 

-1 don’t see the value of any comprehensive exam for government certification. 
However, in the private sector, passing a comprehensive exam is the standard for 
attaining a professional designation. For example, CPA, bar exam, medical boards, etc. 
I’m not sure I see the real value of a comprehensive exam. My experience of being 
around folks preparing for NCMA certification is to just study to take/pass test. I’m not 
sure that requirement alone for NCMA certification equates to an instant professional. 
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- A comprehensive examination is important because it indicates the overall 
knowledge an individual has in contracting. It’s one thing to know about a specific area 
like source selections, but every contracting professional should have a good general 
knowledge of all contracting areas. You have to be able to see the forest through the 
trees. 

Areas for Further Discussion : Please answer the following question. You are not 
confined to the context of one particular certification program; it is possible that none of 
the available certifications contain the optimal requirement. 

1. In what way does requiring the completion of an examination strengthen, or fail 
to strengthen, a professional certification for the contracting workforce? 


End Cycle 2fdk 
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Appendix P. Synopsis of Responses For Delphi Cycle 2d 


_ Table 42. Civilian Responses to Delphi Cycle 2d _ 

In what way does requiring the completion of an examination strengthen, or fail to 
strengthen, a professional certification for the contracting workforce? 

5of6 panelists^ responding^ _ 

The [key] term is “professional”. If this is to be a professional field, education and 
certification is necessary. A certification without a test doesn’t compute as far as I am 
concerned. The test proves the knowledge. Without proof how can you certify a 
professional designation? 

The examination strengthens the value of the certification and is consistent with other 
professions in that regard. As the body of knowledge changes, and it does change 
constantly, the value of the certification changes (diminishes). Re-certification becomes 
a necessary evil once you go down the certification road so you have to anticipate that. 
The more comprehensive the examination, the more value it lends to the certification. 

I stand by my previous statement [during Cycle 1]. The primary benefit of a 
comprehensive examination is it gives credibility to the certification. If there is no exam, 
there is not confirmation that the individual has mastery of the material.. .only that the 
individual was in the classes or was at work for a specified period of time. For the 
certification to have credibility there must be a comprehensive exam and it must be 
possible to fail the exam. The professional certifications granted by AFIT in the past 
aren’t real prestigious. Everybody (well almost everybody) passes the classes and 
attaining the certification isn’t much of an accomplishment. The CPCM exam 
administered by NCMA, on the other hand, had a 60% fail rate in its early days. 
Successfully passing the CPCM is truly an accomplishment and is recognized as such by 
the professional workforce. 

I see no significant way requiring an examination will strengthen the professional 
certification, since the process already requires a combination of education, training, and 
experience. When our professionals reach the highest level of certification.. .they will 
know how to go about executing the infrequent or less familiar [processes]. If there is 
any desire to use examinations, then do it appropriately at the specific training course 
level, or possibly consider the use of examinations in the waiver process only. 

I tend to think that comprehensive examinations need to be incorporated into the APDP 
certification. Class attendance does not prove one is technically knowledgeable of the 
subject matter. Incorporate exams at the following levels: Level I - no exam; Level II - 
an objective multiple-choice exam; Level III - an in depth exam. Comprehensive exams, 
like educational credentials, relate to competence in industry; it should also apply in the 
Government. 
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Table 43. Military Responses to Delphi Cycle 2d 


In what way does requiring the completion of an examination strengthen, or fail to 
strengthen, a professional certification for the contracting workforce? 

4 of 6 panelists, responding _ 

Completion of an examination is essential to the contracting community as we continue 
to build our professional reputation. First, it forces a person into intense preparation for 
the exam. That can only enhance a person’s knowledge. Second, by passing the exam, it 
helps our community identify those who are dedicated to the career field. Third, it 
enhances the contracting community’s reputation for professionalism. Just as the other 
professions have stringent controls on their profession, we must also. 

An examination strengthens the certification by forcing the individual to commit to a 
career field. In addition, it causes the individual to spend a significant amount of time 
thinking about what it is they are involved in, to look at relationships between different 
functional specialties. I sat for two professional examinations: The first and most 
important was the PCO warrant board; that forced me to really learn my craft. The 
second, the CPCM, was an overview of the career field’s current topics. I believe that if 
you’re going to have a single examination that it should be more specific; a la the warrant 
board, and also bring in the general knowledge areas of the CPCM. After all, the goal is 
to demonstrate competence in the career field, not solely knowledge of the world and 
Washington’s latest acquisition streamlining initiative.. .1 think the current 
certification/staffing in the contracting career field is in for rapid change. The next 
generation is being trained in the IPT mode. The “senior functional specialist in the same 
cube” went away nearly ten years ago. The certification exam needs to look at functional 
expertise over the ‘career filed in context’ type of exam that the CPCM relies on. 

I don’t think there is much value added in a comprehensive exam for professional 
certification in contracting. The emphasis should be on acquisition training courses, 
education, and experience. Reading the above comments, what strikes seems to be the 
feeling of shared pain. By making an examination difficult for a level of certification, it 
shows commitment to the profession to pass. I don’t buy that premise. From my 
perspective, my professionalism and growth in my career field is based upon my training 
opportunities and varied experience; not studying some “school answers” to pass a test 
which bears little semblance to what I may be working on or exposed too. 

A good analogy for a professional certification exam is a comprehensive, end-of-degree- 
program exam given by many universities. It’s one thing to take all the courses over the 
years, but have you really retained the information and can apply it. A comprehensive 
test can determine if you still know the information and can apply it. The same goes for a 
required exam for a professional certification. I might have spent 4 years in contracting 
and taken 6 courses, but that doesn’t mean I have retained the information. An exam 
could determine this. It’s very important to determine a certain level of competency 
before giving someone a professional certification. Therefore, an examination would 
strengthen a professional certification for the entire contracting workforce. 
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Appendix Q. Delphi - Cycle 3 


SECTION A - Formal Education 

While most respondents did not indicate a need for a bachelor’s degree to enter the career 
field, there was a high level of agreement that a bachelor’s degree should be required for 
certification. Respondents consistently referenced the “enhancement of professionalism” 
as reason to require a degree before granting professional certification. Although 
Contracting is not presently designated a “professional series” by OPM, the majority of 
respondents believe it is important for contracting personnel to maintain and extend their 
professional standing. 

While agreeing that formal education is valuable to the workforce, and should play a role 
in the certification process, a few respondents did not go as far as requiring a bachelor’s 
degree as a condition for certification. These respondents believe a level of formal 
education should be required, but that a complete degree program is not necessary. 

Recently, the National Defense Authorization Act for 2001 made changes to the 
qualification requirements for contracting personnel. Public Law 106-398 (Section 808), 
amended Section 1724 of title 10, United States Code to read as follows : 


Sec. 1724. Contracting Positions: Qualification Requirements. 

(a) Contracting Officers - The Secretary of Defense shall require that in order to 
qualify to serve in an acquisition position as a contracting officer with authority to 
award or administer contracts for amounts above the simplified acquisition threshold 
referred to in section 2304(g) of this title, a person must - 

(1) have completed all mandatory contracting courses required for a 
contracting officer at the grade level, or in the position within the grade of 
the General Schedule (in the case of an employee) that the person is 
serving in; 

(2) have at least two years of experience in a contracting position; 

( 3 ) 

(a) have received a baccalaureate degree from an accredited educational 
institution authorized to grant baccalaureate degrees, and 

(b) have completed at least 24 semester credit hours (or the equivalent) of 
study from an accredited institution of higher education in any of the 
following disciplines: accounting, business finance, law, contracts, 
purchasing, economics, industrial management, marketing, 
quantitative methods, and organization and management, and 
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(4) meet such additional requirements, based on the dollar value and 

complexity of the contracts awarded or administered in the position, as may 
be established by the Secretary of Defense for the position. 

(b) GS-1102 SERIES POSITIONS AND SIMILAR MILITARY POSITIONS - 

The Secretary of Defense shall require that a person meet the requirements set forth in 
paragraph (3) of subsection (a), but not the other requirements set forth in that 
subsection, in order to qualify to serve in a position in the Department of Defense in - 

(1) the GS-1102 occupational series; or 

(2) a similar occupational specialty if the position is to be filled by a member of 
the armed forces. 

(c) Exception - The requirements imposed under subsections (a) or (b) shall not 
apply to a person for the purpose of qualifying to serve in a position in which the 
person is serving on September 30,2000. 

(d) Waiver - The acquisition career program board of a military department may 
waive any or all of the requirements of subsections (a) and (b) with respect to an 
employee or member of that military department if the board certifies that the 
employee or member possesses significant potential for advancement to levels of 
greater responsibility and authority, based on demonstrated job performance and 
qualifying experience. With respect to each waiver granted under this subsection, the 
board shall set forth in a written document the rationale for its decision to waive such 
requirements. The document shall be submitted to and retained by the Director of 
Acquisition Education, Training, and Career Development. 

(e) Effective Date - This section, and the amendments made by this section, shall 
take effect on October 1, 2000, and shall apply to appointments and assignments to 
contracting positions made on or after that date. 


In other words, civilian employees and military members of similar occupational 
specialties will have to possess a degree, with a major or concentration in business, 
before they are granted entry into the career field, or before they move to a new position 
if they are already in the career field. 

At this point, we can complete our discussion on the subject of formal education. It is 
apparent that certification requirements will have to be at least as restrictive as those 
requirements for entering the career field. Further discussion on the possible effects of 
these changes may be warranted but will have to be reserved for future research efforts, 
as it is beyond the scope of this study. 
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Synopsis: 

- Formal education is valuable to the contracting workforce. Formal education 
enhances professionalism by providing a level of knowledge and exposing individuals to 
the academic process. 

- Contracting personnel must possess a baccalaureate degree in order to enter the 
career field. 

- Upon entering the career field, contracting personnel must meet additional 
requirements, including the achievement of a level of experience and the attendance of 
designated PCE courses, before being granted a contracting officer’s warrant or 
professional certification. 

Question 1 : Do you have any additional comments regarding the discussion 
summary and synopsis above? 


SECTION B - Levels of Certification 

Many respondents support the current 3-tiered APDP certification process, but some are 
in favor of a single-level certification. Based on our discussion to date, the following 
comments present the group’s support and criticism of multiple levels of certification: 

Support 

- Certifying at progressive levels provides intermediate goals for individuals entering 
the career field.. .increases commitment by building confidence along the way. 

- Certification at graduated levels helps identify an individual’s proven ability to 
date.. .helps pair the individual with an appropriate duty position. 

- Certification at multiple levels aids employers in determining a potential employees 
level of motivation and commitment.. .individuals with a certain degree of 
experience and training can be expected to attain a certain level of certification. 

- Multiple-level certification mirrors the natural progression of individuals... 
personnel begin as apprentices and eventually become contracting masters. 

Criticism 

- Certifying individuals at multiple levels decreases the credibility and prestige of the 
certification.. .offers a “consolation prize” to those lacking the ability to achieve 

top 

certification honors. 

- It is difficult to determine the certification level that should be required for a given 
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duty position since so many contracting positions require duties of widely varying 
levels of difficulty. 

- True professions reserve certification for the most highly qualified personnel... 
clients do not want to be served by a lower level CPA, doctor, or lawyer.. .the title 
instills confidence and credibility because it is difficult to obtain. 


Question 2: How many levels (1,2,3, or more) should be included in a professional 
certification program for contracting personnel? Please provide support for your 
answer. 


SECTION C - CO Warrant vs. Certification 

One interesting issue arising from the first two cycles is the similarity between the 
requirements for obtaining a contracting officer warrant and those for certification. In 10 
USC 1724, we see that individuals must meet education, experience, and training 
requirements to obtain a PCO or ACO warrant; likewise, certification requires a level of 
education, experience, and training. Based on the changes enacted by P.L. 106-398, 
individuals will enter the career field with a level of education and then progress in their 
training and experience to obtain a warrant or certification. 

Question 3: Should an individual be required to qualify for a warrant before being 
granted certification? At what level (for those supporting multiple-level 
certifications)? 


Question 4: If not, should an individual be required to possess a certification before 
being granted a warrant? At what level (for those supporting multiple-level 
certifications)? 


SECTION D - Trainine 

Many respondents indicated that the current PCE training is inadequate. Most 
respondents stated that the level of difficulty at in-residence courses does not guarantee 
the attendee graduates with the desired degree of training. At the same time, most 
respondents still favor in-residence coursework over computer based training (CBT). 
One respondent recommended using CBT to furnish a level of knowledge before 
allowing an individual to attend an in-residence course. 
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Question 5: Regarding the number of certification levels you selected in Question 2, 
what PCE courses or subject areas should be required at each level? 


Question 6: Should all personnel be required to attend the same courses, or should 
a variety be offered for each subject area? 


Question 7: What format of PCE (CBT, In-Residence, VTC,...) should be required 
at each level? 


SECTION E - Experience 

Most respondents indicated the current certification requirements for experience were 
problematic. While all respondents agree that varied experience is desirable, most did 
not suggest that a broad base of experience be a requirement because such a requirement 
would be very difficult to measure. Several respondents suggested using a method such 
as the enlisted OJT record, or Copper Cap training record, to ensure the quality of the 
experience obtained. Most respondents agree with using the measure of “years” to 
indicate an individual’s experience for certification, but these same respondents are 
concerned that the experience be relevant - what specific training occurs within a time of 
experience? One respondent did not agree with the measure of “years” and stated the 
experience requirement should be monitored by developing and tracking specific duty 
tasks at each level of certification. 

Questions 8: Using the number of levels of certification you selected in Question 2, 
what specific experience requirements do you propose at each level? 


Question 9: How would you measure this experience to ensure the appropriate level 
of quality and relevance of the experience? 


SECTION F-Waivers 

Question 10: Which, if any, of the above requirements for certification would you 
be willing to waive? Under what circumstances? 


151 




SECTION G - Examination / Re-certification 

Many respondents stated that a comprehensive examination increases the credibility of a 
certification program. Others asserted that individuals can be “taught the test” and that it 
is not a true indicator of professional development. At this point, there is no definable 
group opinion on the issue. 

While many respondents agree that requiring re-certification ensures individuals continue 
to progress in their training and education, several respondents did not find value in the 
requirement. Some respondents state that requiring certification at multiple levels 
alleviates the need for re-certification. Others believe that individuals should be required 
to attend continuing education but that a formal reissue of the certification is not 
necessary. Two respondents indicated that the re-certification requirement is actually of 
more benefit to the certifying associations (keeps them in business and funded) than to 
the workforce. On 15 Dec 98, USD (A&T) issued a mandatory Continuous Learning 
(CL) policy for the DAWIA acquisition workforce. 

Question 11: At what level would you require individuals to pass a comprehensive 
examination before granting certification? 


Question 12: Should an individual be allowed to retain their certification status if 
they do not meet the requirements of the Continuous Learning policy? 

Question 13: What other requirements should be met in order for an individual to 
maintain their certification status? 


End Cycle 3. 
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Appendix R. Synopsis of Responses For Delphi Cycle 3 


Table 44. Responses to Delphi Cycle 3 / Question 1 

Do you have any additional comments regarding the discussion summary and 
synopsis above? 9 oj_ 12 panelists^ responding _ 

While I agree a bachelor’s degree should be required to get higher levels of certifications, 
a bachelor’s degree shouldn’t be required to enter into the career field. Or, if it should be 
required, the degree requirement should be gradually implemented with experience being 
an acceptable alternative. I’m particularly sensitive to this because in my location it’s 
very difficult to hire people who have degrees and 24 business hours. I think it’s 
unreasonable to expect one day to not have to have a degree and the next day to have a 
degree. 

Yes, I seriously disagree with Public Law 106-398 (Section 808), amended Section 1724 
of title 10, United States Code which states must have undergrad degree before entrance 
into the career field. The primary conduit for enlisted membership into the operational 
contracting field is through cross training. The vast majority of these highly motivated 
and sharp individuals don’t have a degree. The ability for AF contracting to meet 
wartime contingency requirements will be severely compromised. SAF/AQC has 
recognized this problem and is working a request for waiver to the provision for enlisted 
personnel. 

I strongly believe that a degree is should be a requirement to enter the contracting career 
field. We have come to a point in the history of our career field that we need to turn the 
comer, take a stand, and make a degree the minimum entry requirement and build a 
successful career from there. We have been heading in the direction for the last 15 years 
or so. Let’s continue that push. The one exception I would make would be for military 
enlisted. But in their case they will be specifically designated to deploy and should focus 
their training to meet that requirement. 

Yes, the staff officer at AQ that let the language in the new Public Law slip through 
without bringing it to the attention of AQC [should] be shot. 

5 of the experts did not offer additional comments. 
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Table 45. Responses to Delphi Cycle 3 / Question 2 


How many levels (1, 2, 3, or more) should be included in a professional certification 
program for contracting personnel? Please provide support for your answer. 

9 of 72 pamlists_ responding^ _ 

The certification program should be 3 tiered. This provides enough little steps before 
getting the highest certification. It’s important to have multiple steps to give people 
something to short for in the short term. If someone has to work 6 years to final get their 
first certification, you will probably lose a lot of people along the way. We have a bad 
enough retention/recruiting problem with our civilians and military and we don’t need a 
certification program to make the problem worse. 

The present DoD APDP graduated approach of three levels is effective. It rewards 
progressively the training and experience already completed. It further combines 
additional training and experience to provide the eventual result of a “contracting 
master.” 

I’m comfortable with the current three-tiered level of certification. This applies well 
within the different grade structures and responsibility associated with Gov. personnel 
system. Additionally, a single certification level for non-governmental personnel is also 
appropriate. It standardizes the level of expertise and knowledge across a spectrum 
(corporate arena), which doesn’t have standardization. 

I think the 3-level approach is a good one if the requirements for the levels are consistent 
and strictly adhered to. 

After reading some of the comments above, I tend to agree we need to reserve the 
certification for the most qualified people. This will make it less confusing to those 
outside the contracting career what the certification means. I would also make getting a 
CPCM part of the certification. In other words, once a person becomes certified it means 
they are the absolute tops in the contracting career field and are entitled to man our most 
critical contracting positions. Levels one and two do really indicate much within the 
career field, and it certainly does not mean anything to those outside the career. 

Three seems to be a good number. I believe people should be allowed to advance within 
their capacity. Having different levels give people incentive to continue to strive for 
advancement. 

Concur with the statement that certification should be reserved for the highest level and 
therefore only one level of certification should exist. Having the second level of 
certification truly serves as a “consolation prize” for those either lacking in ability or 
motivation to attain the “professional” designation. The second level certification 
“CACM” is nothing more than a block checker for true professionals. When I look at 
qualifications and I see someone who is eligible for CPCM and has only attained CACM, 
I consider it a negative not a plus in evaluating the individual. 
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Table 45. (Continued) 


How many levels (1, 2,3, or more) should be included in a professional certification 
program for contracting personnel? Please provide support for your answer. _ 

I disagree with [two of the] points above. There are differences in the professions. 
Doctors have specialties, as do lawyers. They achieve advanced degrees, there is a 
hierarchy based upon past positions held, etc. We, as laymen do not see those 
distinctions since we are outside the career field. However, it is widely known and 
acknowledged that not all lawyers can argue before the Supreme Court, or all doctors can 
operate on your brain. In addition, it is well known who the doctors, lawyers, etc. are; 
but it’s within the community. Yet, among doctors all have MD after their name and 
among lawyers they all use Esq. 

Bottom line, stick with the three levels. The supporting comments are right on. The need 
to differentiate in experience and training, especially, in the early years of contracting 
training is important. Lastly, consider that we will be facing a huge influx of new people 
in the next five to ten years. We’ll need to stratify them somehow and sort the fast 
movers, average and slow. 

A single certification level is optimal. I tie certification to the same status as being 
granted an unlimited CO warrant. The warrant is the license to "operate" in the 
contracting world. It is granted (hopefully) only to those who have achieved the 
knowledge and experience necessary to function on behalf of the federal government. 
Similarly, one is certified as capable of operating at the ultimate level, or not. Lesser 
gradations are misleading and unnecessary. 
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Table 46. Responses to Delphi Cycle 3 / Question 3 


Should an individual be required to qualify for a warrant before being granted 
certification? At what level (for those supporting multiple-level certifications)? 

9 of 12 panelists, responding^ _ 

Three experts stated that certification should come before a warrant, supporting their 
answers with comments included in Table 46. 

No, you shouldn’t tie the ability to get certified at different levels with the qualification 
for a warrant. While there is a close relationship between the two actions, in reality, you 
cannot qualify for a CO warrant without having met the minimum requirements for Level 
II certification. The qualification and granting of a warrant should be based upon 
organizational needs and workload. 

No. A person should be able to obtain the highest certification without having to be a 
CO. However, in granting certifications being a CO should play a role. We still can 
maintain high certification standards without having a person being a CO in order to 
obtain a warrant. 

No - Level I should be a training ground - no warrant at this level. 

No, I don’t believe so. There are some positions such as procurement analyst positions 
and price analyst positions that do not require a warrant. The individuals in these 
positions can still have the expertise and knowledge of a warranted individual and they 
should have the opportunity to gain the professional certification. 

I believe these are different issues. Warrants are issued based upon need. Certification is 
issued based upon qualification. I may need a highly qualified contracting person but not 
a warrant. Where the confusion arose is that in most of our training days to be granted a 
warrant we had extensive OJT in preparation for a warrant board. But, these are different 
and you’d never get a good warrant board to meet just to determine if a person was 
qualified and then not issue the warrant. 

Yes, absolutely. And only after a rigorous examination conducted through and by the 
individual's supervisory chain. 
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Table 47. Responses to Delphi Cycle 3 / Question 4 

If not, should an individual be required to possess a certification before being 
granted a warrant? At what level (for those supporting multiple-level 
certifications)? 8 of 12 panelists, responding. _ 

A certification should come before a warrant. A certification shows an individual's 
ability to meet some minimum requirements. In many cases, these minimum 
requirements and certification allows an individual to be hired into a specific job. This 
specific job will then require an individual to get a warrant. An individual should be able 
to get a warrant after they obtain their first certification. The warrant should be 
commensurate to their certification. For instance, an individual who just obtained their 
first certification should be given some type of limited warrant (i.e., small purchase, etc.). 
Once an individual has obtained their 2 nd and 3 rd certifications, then they should be given 
more authority in their warrant. 

Yes. I think the certification should come before the warrant. Using the multiple-level 
approach; require Level I for warrants up to SAP and level II for warrants above the SAP. 
No difference for ACO or PCO warrants. 

Yes, at a minimum, the person should have a Level II certification. 

I think the type of Contracting Officer warrant should be tied to the certification level. 
Only Level Ills should have unlimited warrants. A small purchase warrant would be 
appropriate for a Level 1 certification. 

No. Assuming going to a single level certification, a person should be able to obtain a 
warrant without having the certification. However, being a CO should be an important 
part of getting a certification. 

Yes. If there are three levels. Level II could be the starting place. Some commands have 
limited warrants. Level II would be a good place for a limited warrant. 

No. The warrant is an on the job thing and indicates fitness for current assignment. The 
professional certification is an over and above and in addition to indicating fitness for 
current assignment, indicates individual development and motivation to advance beyond 
the current position. 

In the DCMA a person must be certified level II before being granted a warrant. I think 
ESD and ASD did very well in granting their warrants before the certification business 
began. Other advanced contracting shops didn’t board their people as well, which led to 
the certification process. So, you don’t want to return to that kind of problem. By 
requiring a cert before granting a warrant you at least screen out the people unable to 
walk and chew gum. 
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Table 48. Responses to Delphi Cycle 3 / Question 5 


Regarding the number of certification levels you selected in Question 2, what PCE 
courses or subject areas should be required at each level? 

9 of 12 panelists, responding _ 

If we take the wisdom of congress and section 808 as our guide, education and training 
akin to 24 hours of business would be a good starting place. Certainly contract law, 
contracting fundamentals, quantitative studies (read business statistics) and financial 
analysis are core courses that should be included. I also believe economics (little and 
large) and purchasing management are useful. 

The current number of courses you have to take to get your APDP certification is 
adequate for each level of certification. 

Use the present courses, but move law to Level I and add an elective for Level II: 

Level I = Basics of Contracting, Principles of Contract Pricing, and Contract Law 
Level II = Intermediate Contracting, Intermediate Pricing, and 1 elective 
Level III = Executive Contracting and Management for Contracting Supervisors 

I’m comfortable with the current breakdown of courses for the 3 levels of APDP 
certification. 

I believe the course workload that is currently required is a good balance of basic skills at 
all levels if the employee is coming into the career field with a BA in Business. 

Assuming one level for certification the following courses (or related course) should be 
required: CON 301 Executive Contracting; Acquisition Planning and Analysis; Program 
Management; Contract Admin (basic, intermediate, and advanced); Basic Contracting 
(intermediate and advanced); Negotiation workshop; Basic Pricing; Contract Law. I look 
at this as a long-term process that will take many years to reach the one level of 
certification. Of course this is in addition to having a four-year degree. I would even 
make getting a Masters degree part one obtaining the certification. Again, the 
certification should show the world that this person is one of the best in our career field. 

First of all I still think classroom training is far superior to CBT. The eye-to-eye, give 
and take of a classroom increases learning. Level I - all basic FAR, Law. Level II - 
advanced learning in Law, Fiscal law, some management classes. Level III - advanced 
management; offer some electives that include other disciplines that may or may not be 
similar to 1102. 
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Table 48. (Continued) 


Regarding the number of certification levels you selected in Question 2, what PCE 
courses or subject areas should be required at each level? _ 

I don’t think the certification should be based on PCE courses attended/passed. I feel the 
certification should be based on a comprehensive exam and experience. If you pass the 
exam and have the requisite years of experience, you should be certified whether or not 
you have attended PCE courses. Likewise if you’ve attended all the PCE courses in the 
world and have not mastered the material well enough to pass the exam, you should not 
be certified. I fell the requirement for a minimum number of years experience is 
necessary to demonstrate that the individual has shown some ability to practice that 
which they learned in an academic environment. Knowing what to do in a detached 
academic/exam environment and actually recognizing the situation in real life and 
analyzing alternative solutions to solve a real problem are different skills. 

Well, are we talking about base contracting, systems or ACO work? Probably a generic 
so.. .for a Level I how about a basic contract law course, a basic finance course, a basic 
contract admin course and a base level services course (to include a bit on A-76 studies). 
For Level II: two courses in pricing. The first centering on RFP/IFB preparation in both 
the commercial and the negotiated environment. A second pricing course focused on 
contractor developed rates and factors. Add a course on contract types, and an advanced 
contract administration course including a sections on the relationships between CLIN 
structures, SPS, DFAS and contract close outs. For Level III a course on program 
management, a course on issues in contract law and a course on current issues and trends 
in acquisition. 
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Table 49. Responses to Delphi Cycle 3 / Question 6 


Should all personnel be required to attend the same courses, or should a variety be 
offered for each subject area? 9 of 12_ panelists^ responding _ 

Each personnel should attend the same courses. The certification validates minimum 
qualifications of an individual. The only way to determine that everyone has minimum 
qualifications is to have them all take the same courses. 

There should be a required core curriculum for all individuals (must include the four 
courses mentioned above) and electives for the rest. 

All should attend the same courses with the obvious exception of the elective course in 
Level II. 

One of the biggest complaints I’ve heard from my folks is the non-applicability of 
portions of the course. For example, case studies or course examples are built around 
weapon systems style buys or issues. For a large number of students from the operational 
contracting arena, the training lessons are lost because they don’t see the applicability. 
Maybe there should be shreadouts of courses, which concentrate on systems or 
operational concerns. 

It would be great if there were a large selection of courses to choose from, but from a 
practical matter it would turn into an administrative nightmare. It is hard enough getting 
the minimal amount of courses scheduled, all databases updated and correct people get 
the proper credit now. 

Certification requirements should be specific as to what courses are needed and what 
would be allowed as a substitute. As long as there are strict guidelines, substitutes can be 
allowed. 

Depending on the class - there should always be some basic classes, like FAR. However, 
as stated in 5 above a group of electives would be great. 

Since I don’t believe any courses should be mandatory, I would have to pass on this 
answer. If you need a response my logical response given my stand on course work is 
that a variety of courses should be acceptable. 

There should be a certain minimum standard that all have to take. Would be nice if 
funding were available for people to take in-resident elective courses - that will be a 
tough sell however. I really think that a standard should be maintained to ensure 
consistent minimum standards across the department. 


160 




Table 50. Responses to Delphi Cycle 3 / Question 7 


What format of PCE (CBT, In-Residence, VTC,...) should be required at each 
level? 9 of 12 panelists^ responding _ 

In residence for the basic courses. Others can be taught via other means. 

For the core courses, the PCE format should be in-residence. CBT or VTC cannot 
simulate class interaction. The interaction between classmates and instructor is so 
important. 

Level I = Basics of Contracting (In-Residence), Principles of Contract Pricing (CBT), and 
Contract Law (VTC) 

Level II = Intermediate Contracting (In-Residence), Intermediate Pricing (CBT), and 1 
elective (method of instruction dependent on the course) 

Level III = Executive Contracting (In-Residence) and Management for Contracting 
Supervisors (VTC) 

The greatest source of learning comes from hearing real-life experiences and lessons 
learned from the instructor or fellow classmates. I endorse in-residence for all 
certification level courses. 

Even though it is an expensive proposition, the most important learning experience is in 
residence. The learning from one another and the networking system one gains by going 
TDY to a course is just as important as the content of the course taught. 

I prefer in-residence for all courses. But at least, the executive courses must be in 
residence. The benefit comes from meeting other professionals in the career field. 
However, for some of the basic course we should allow them to be taken by 
correspondence. 

As stated above, I prefer in-residence. However, there could be a mixture. More VTC 
than CBT. Level 1 needs to be in-residence to aquatint new people to others ideas. Level 

II is not as critical. However, for advanced classes in Level III - interaction is very 
important. 

Again, I don’t believe that course work should be required. I do believe it should be 
encouraged and individuals made aware that completing the coursework will improve 
their knowledge and thus their chances of successfully completing the exam. However, I 
believe the exam should be so tough your chances of passing it without the PCE are very 
slim. 

These courses should all be in resident. The learning comes from the interaction of the 
people; the CBT stuff isn’t really working. 
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Table 51. Responses to Delphi Cycle 3 / Question 8 


Using the number of levels of certification you selected in Question 2, what specific 
experience requirements do you propose at each level? 9_ of 12 panelists responding 

For one level of certification I would opine that a minimum of seven years experience is 
optimal. Waiverable by local authority on case-by-case review. 

Level 1-4 years; Level II - 6 years; Level III - 10 years 

I would increase the number of years because so much of what you need to know in 
contracting is based on experience. 

Use the present experience requirements. 

I don’t think it’s unreasonable to establish a method such as enlisted OJT or Copper Cap 
to identify a specific knowledge base and task familiarization for each level. These could 
be identified readily with the proper resources, which I don’t currently have. 

Level 1-2 years; Level II - 6 years; Level III - 10 years; these are all minimum amounts 
of experience for each level. 

Again, I’m tough on what the requirements should be. I believe in order to identify the 
tops in our career field, we must require years of experience, but they should be in 
different kinds of jobs. For example, so many years in systems, so many years at the HQ 
level, so many years in operational contracting, so many years in admin, etc. Again, we 
want to identify the leaders and decision-makers in our career. Someone who stays in 
one location will never obtain a certification. This is not a put down on the people who 
stay in one area. We need that. But those who are willing to grow and meet new 
challenges are the ones who get certified. We are looking at the top 20% or so who will 
eventually earn a certification. And that’s OK. And that’s why a certification will mean 
something. 

Level I - little experience; Level II - 2 to 3 years; Level III - at least 5 years. 

I think there should be one level of certification and the level of experience should be 
seven years. Why seven? Because that’s my judgment on how long it takes someone to 
be well versed in the total contracting career field. 

One thing I would NOT require at any level is a source selection, there simply aren’t that 
many major ones around anymore. In fact, I’m not sure this is an answerable question. If 
we say that a person needs to have worked a protest, or done a contract restructure then 
we have to ensure we screw up a contract award so we generate a protest. The list clearly 
goes on. What if a person is working as an ACO and never writes a P mod? What if the 
person is a PCO and never negotiates an overhead rate? Someone smarter, with more 
time has to answer this one for you. 
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Table 52. Responses to Delphi Cycle 3 / Question 9 


How would you measure this experience to ensure the appropriate level of quality 
and relevance of the experience? 9 of 12_panelists responding _ 

Local authority has final decision based on review of jobs held, performance levels, years 
in grade and other factors. No "one size fits all" rules. 

One good way is the ways the enlisted sign off on their skill levels. In order to be a 3- 
skill level, you should be familiar with a certain number of tasks. The same goes with 5, 

7 and 9 skill levels. You could do the same thing by saying if you have 4 years in 
contracting and looking to get your 1 st level certification you should be familiar with the 
following areas (source selections, modifications, protests, etc.) You would need an 
individual at a higher skill level to sign off saying this individual's knows all these skills. 

Define qualifying experience as equivalent to GS-1102 duties with increasing 
responsibilities from SAP for level I to procuring and administration beyond SAP for 
higher levels. 

It would be up to the training certifier to ensure the identified tasks were completed and a 
sufficient level of performance accomplished to “sign-off’ the task. 

To measure this experience in calendar years is probably the only way. 

By the number of years in different kinds of jobs as discussed above. 

By time in service and relevance of experience. If being hired from outside the 
government relevance of experience could be used. 

I don’t think there is a way to measure appropriate level or quality for relevance. Any 
criteria you come up with can be worked around with some determined pencil whipping 
and puffery when responding to the application questions. Therefore I suggest the only 
criteria be length of service with a belief that if you’re involved in the process for seven 
years at the GS-7 and above level, something had to stick. 

I go back to the training folders required of all SAC contracting management officers. It 
was a good idea and it was reviewed during ORIs. A similar item could be prepared, 
obviously not all items could be completed (see response to question 8 above) but many 
items could. 
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Table 53. Responses to Delphi Cycle 3 / Question 10 

Which, if any, of the above requirements for certification would you be willing to 
waive? Under what circumstances? 9 of 12 panelists, responding _ 

Waive time requirement for the true fast burners. No waiver of education and training 
but allow interpretation of what counts as applicable. Must be local decision(s). 

I would be willing to waive a degree requirement if an individual has an appropriate 
number of years of experience. 

Only education similar to the waiver requirement found in “Sec. 1724. Contracting 
Positions: Qualification Requirements.” “The acquisition career program board of a 
military department may waive education requirements with respect to an employee or 
member of that military department if the board certifies that the employee or member 
possesses significant potential for advancement to levels of greater responsibility and 
authority, based on demonstrated job performance and qualifying experience.” 

Waive the education requirement for enlisted. 

The only waivers should be for hard to fill positions, which require a certain certification 
level. The waiver would only be good for the time the person is filling that position. 

None. I want a very challenging process with strict requirements in order for a person to 
get certified. 

I would not waive the requirement for experience time. Because time is the only criteria 
I would use on experience, I think it should be totally inflexible. I would be flexible in 
evaluating time in relevant acquisition positions other than just contracting. For instance, 
if an individual was not an 1102 for seven years but had served as a program manager for 
a significant program, not just a project, I would like to be able to review and waive the 
experience requirement. 

I’d be willing to waive just about anything. I just hired a guy as an engineer. He used to 
be my contract lawyer (he didn’t like lawyers.. .go figure), yet I’m currently required to 
send him to ACQ 201. Doesn’t make sense, waivers can’t be given out lightly but they 
should be available. 

One expert simply stated, “No waivers. ” 
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Table 54. Responses to Delphi Cycle 3 / Question 11 


At what level would you require individuals to pass a comprehensive examination 
before granting certification? 9 of 12panelists responding _ 

Coincident with the warrant board examination and warrant granting decision. 

I would have them pass an exam at every level of certification. 

Depends on what certification you are talking about. If you are talking the current 3 level 
APDP system, then not at any level. A non-governmental org., such as NCMA, has 
established a comprehensive exam policy for its certification. 

Level III certification should have a comprehensive examination tied to it. 

Whenever the person is ready. However, by natural selection, a person will not be ready 
until taking courses and spending a certain amount of time on the job. Just like getting a 
CPCM, a person can take the test at anytime, but in order to have a chance to pass the test 
that person will have to have a certain level of knowledge, which can only come from 
taking courses and spending years in contracting. 

I agree that certification at multiple levels is all that is necessary - no test. 

I think there should only be one certification level for a professional designation. I have 
no problem with the three-level certification for qualification for positions in place today 
in DoD. 

Two other experts indicated they would not require an examination at any level. 
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Table 55. Responses to Delphi Cycle 3 / Question 12 

Should an individual be allowed to retain their certification status if they do not 
meet the requirements of the Continuous Learning policy? 

9 of 12 panelists responding _ 

No, I think it’s important to keep current in the career field. The career field has changed 
so much in the last ten years that if you didn’t continually learn you shouldn’t have your 
certification. 

Yes. Can’t remove certification unless we can clearly ensure the individual will be 
offered the training opportunities through their work centers. 

No, the acquisition world changes so much, CL is mandatory to stay current. 

No - keeping current with the changing acquisition field should be a requirement of the 
certification level. 

No. Maybe place them in an “inactive” status until they complete the requirements to re¬ 
certify. The continuous learning requirements are not that tough to maintain. 

No the continuous learning should be required to maintain certification. Without the CL 
requirement, there is no way of insuring the individuals do not stagnate. 

Yes, look at me. Courses were available for my level of expertise when I was [in a 
MAJCOM-level staff position]. Since then, my organization hasn’t had the funding and 
the courses are not there if the money was. Also, one unique thing about this career field 
is this: I knew more about contracting as a [previous grade] than I do now as a [current 
grade]; but, I know more about managing an acquisition organization now than I did then. 
As you go up in grade you become less technical and more a leader. If you do stay a 
technician you tend to narrow your expertise, you become an expert on one or two 
subjects because you’re the staff guy assigned. But, you lose lock on all the other 
sections of the FAR. So this question needs to keep that in mind. 

Two other experts simply stated, “No ”. 
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Table 56. Responses to Delphi Cycle 3 / Question 13 


What other requirements should be met in order for an individual to maintain their 
certification status? 9 of 12 panelists responding- _ 

Local supervisory chain must agree to extending certification (this covers a realm of 
issues including performance measures, effectiveness, work habits, personal 
relationships, and a host of etc). 

Revaluation of experience and relevancy 

Continuous learning should be the only requirement. As I stated earlier, evaluation the 
quality or relevance of experience is too hard to do and can be too easily defeated by 
creative writing to be of any significant value. 

Question assumes I agree that certification should be reexamined on a periodic basis. I 
don’t therefore I only think working in the appropriate jobs is necessary to maintain 
certification. 

All other respondents indicated no other requirements are necessary. 
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Appendix S. Delphi - Cycle 4 - Recommend Changes to Current Programs 


In the two sections that follow, I have listed the current certification requirements for the 
APDP and NCMA (CPCM) programs. Please review these requirements and recommend 
any changes you believe could improve the programs. 

Section I : Acquisition Professional Development Program (APDP) 


Current Requirements . 


CAREER 

LEVEL 

EDUCATION 

EXPERIENCE 

TRAINING 

Entry 

Level (I) 

Have ONE of: 

Baccalaureate Degree; 

1 year in 
contracting 

Basics of 
Contracting 
(CON 101) 


24 Semester Hours accounting, law, 
business finance, contracts, 
purchasing, economics, industrial 
management, marketing, quantitative 
methods, or organization and 
management 


Principles of 
Contract 

Pricing(CON 104) 


10 years contracting experience as of 1 
Oct 91 



Intermediate 
Level (II) 

Same as Level I 

(Desired) Graduate studies in business 
administration or procurement 

2 years in 
contracting 

(Desired) 2 
additional years 
in contracting 

Intermediate 
Contracting 
(CON 202) 

Intermediate 

Contract 

Pricing(CON 204) 




Government 
Contract Law 
(CON 210) 

Advanced 
Level (III) 

Same as Level I 

(Desired) Masters degree in Business 
Administration or Procurement 

4 years in 
contracting 

(Desired) 4 
additional years 
in contracting 

Executive 
Contracting 
(CON 301) 

Management for 
Contracting 
Supervisors 
(CON 333) 


Note: Education requirements will likely change to require a baccalaureate degree AND 24 semester 
hours in business (based on the recent legislation) 
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Recommended Changes . Please consider the comments generated during the previous 
cycles of discussion and then indicate the changes you now believe necessary to improve 
the APDP certification program. 

1. Number of Levels: 

2. Education: 

(a) Level I : 

(b) Level II : 
tc) Level III : 

3. Experience: 

(a) Level I : 

(b) Level II : 
tel Level III : 

4. Training: 

(a) Level I : 

tbl Level II : 

(c) Level III : 

5. Comprehensive Examination (Not a current requirement): 

6. Re-certification (Not a current requirement): 
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Section II: NCMA Certified Professional Contract Manager (CPCM) 


Current Requirements . 


FORMAL 

EDUCATION 

RELEVANT 

EXPERIENCE 

TRAINING 

EXAMINATION 

Bachelor’s Degree 

2 years experience in: 

8 non-credit, 

Completion of 


procurement, legal, MIS/IT, 
inventory mgt, project mgt, 
production, R&D, 
marketing, QA, finance, 
business mgt 

certificate, or 
continuing education 
courses 

CPCM exam 


Note. Formal education requirements are waiverable on a case-by-case basis if the 
individual possesses at least 2 years of college and 10 years relevant experience, while 
the experience requirement is waiverable if the individual possesses a bachelor’s degree. 


The CPCM examination is an all essay test consisting of two sessions. The first session 
tests the individual’s knowledge of general contracting principles, while the second 
session is concentrated in seven specific areas: legal; finance, economics, and 
accounting; production; contracting; logistics management; commercial purchasing; and 
state and local government procurement. 

Re-certification is required every five years through the completion of a total of 60 hours 
of instruction, 10 of which must be within the 18 months prior to re-certification. If this 
continuing education is met within the five-year deadline, the examination need not be 
retaken. 

Recommended Changes . Please consider the comments generated during the previous 
cycles of discussion and then indicate the changes you now believe necessary to improve 
the CPCM certification program. 

1. Number of Levels (Only one level of CPCM certification is offered; however, a 
separate certification (Certified Associate Contracts Manager (CACM)) is also 
available ): 

2. Education: 

3. Experience: 

4. Training: 

5. Comprehensive Examination: 

6. Re-certification: 

End Cycle 4 . 


170 





Appendix T. Synopsis of Responses For Delphi Cycle 4 


Table 57. Recommended Changes to Air Force APDP Certification Requirements 


Certification 

Element 

Recommended Changes 

Number of 

No Change (9) 

Levels 

Two levels- adopt the NCMA CACM and CPCM certifications as 
official DoD programs (1) 

Formal 

Level I: No Change (6) 

Education 

Require Bachelor’s Degree (2) 

Require Bachelor’s Degree w/ 24 business hours (2) 

Level II: No Change (5) 

Require Bachelor’s Degree (1) 

Require Bachelor’s Degree w/ 24 business hours (2) 

Require Bachelor’s and some graduate study (1) 

Require Bachelor’s; Desire Master’s Degree (1) 

Level III: No Change (3) 

Require Bachelor’s Degree (1) 

Require Bachelor’s; Desire Master’s Degree (3) 

Require Master’s Degree (3) 

Experience 

Level I: No Change (8) 

Two years contracting experience (2) 

Level II: No Change (5) 

Four years contracting experience (3) 

Five years contracting experience (1) 

Six years contracting experience (1) 

Level III: No Change (4) 

Six years contracting experience (2) 

Eight years contracting experience (1) 

Ten years contracting experience (3) 
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Table 57. (Continued) 


Certification 

Element 

Recommended Changes 

Training 

Level I: No Change (8) 

Same classes; increase level of difficulty (1) 

Add Contract Law course to Level I (1) 

Level II: No Change (9) 

Add elective (1) 

Level III: No Change (10) 

Examination 

Do not add requirement for examination (6) 

Require comprehensive examination - no specific level indicated (2) 

Require successful completion of comprehensive examination for 

Level III certification (2) 

Re-Certification 

Do not add requirement for re-certification (6) 

Re-certify every two years by completing at least 40 hours of PCE 
per year (1) 

Re-certify every six years by completing at least 40 hours of PCE 
per year (1) 

Adopt the NCMA re-certification standard (1) 

Require re-certification for certification levels II and III - no specific 
course requirements indicated (1) 

Numbers 

in parentheses indicate number of experts making recommendation 
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Table 58. Recommended Changes to NCMA CPCM Certification Requirements 


Certification 

Element 

Recommended Changes 

Number of 

Keep single level program (9) 

Levels 

Add a higher level certification for those with advanced degrees (1) 

Formal 

Education 

No change (8) 

Require Bachelor’s degree with no exception (2) 

Experience 

No Change (5) 

Four years relevant experience (2) 

Five years relevant experience (2) 

Six years relevant experience (1) 

Training 

No Change (9) 

Add a requirement for applicant to have had a Contracting Officer 
warrant or an equivalent representation of responsibility (1) 

Examination 

No Change (9) 

Offer exam on demand at any time instead of only twice 
per year (1) 

Re-Certification 

No Change (10) 

Numbers 

in parentheses indicate number of experts making recommendation 
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